-- Logs begin at Mon 2022-10-24 12:34:06 UTC, end at Mon 2022-10-24 13:51:12 UTC. -- Oct 24 13:06:31.524603 dv-cinder-862491-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 24 13:06:31.533111 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 24 13:06:31 2022] *** Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: nodename: dv-cinder-862491-1-iscsi Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: machine: x86_64 Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: clock source: unix Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: pcre jit disabled Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: detected number of CPU cores: 4 Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: current working directory: / Oct 24 13:06:31.535103 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: detected binary path: /usr/bin/uwsgi-core Oct 24 13:06:31.535811 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: your processes number limit is 47680 Oct 24 13:06:31.535811 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: your memory page size is 4096 bytes Oct 24 13:06:31.535811 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: detected max file descriptor number: 2048 Oct 24 13:06:31.535811 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: lock engine: pthread robust mutexes Oct 24 13:06:31.535811 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: thunder lock: enabled Oct 24 13:06:31.535811 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 24 13:06:31.535811 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 24 13:06:31.574858 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: Python main interpreter initialized at 0x55e8118639c0 Oct 24 13:06:31.574858 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: python threads support enabled Oct 24 13:06:31.574858 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: your server socket listen backlog is limited to 100 connections Oct 24 13:06:31.574858 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: your mercy for graceful operations on workers is 90 seconds Oct 24 13:06:31.575354 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: mapped 403077 bytes (393 KB) for 2 cores Oct 24 13:06:31.575354 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: *** Operational MODE: preforking *** Oct 24 13:06:31.575591 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: *** uWSGI is running in multiple interpreter mode *** Oct 24 13:06:31.575591 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: spawned uWSGI master process (pid: 81591) Oct 24 13:06:31.575759 dv-cinder-862491-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 24 13:06:31.580326 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: spawned uWSGI worker 1 (pid: 81603, cores: 1) Oct 24 13:06:31.580326 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: spawned uWSGI worker 2 (pid: 81604, cores: 1) Oct 24 13:06:31.580326 dv-cinder-862491-1-iscsi devstack@keystone.service[81591]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 24 13:06:35.884064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 13:06:35.884064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: from cryptography.utils import int_from_bytes Oct 24 13:06:35.884962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 13:06:35.884962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: from cryptography.utils import int_from_bytes Oct 24 13:06:35.886051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 13:06:35.886051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: from cryptography.utils import int_from_bytes Oct 24 13:06:35.886161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 13:06:35.886161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: from cryptography.utils import int_from_bytes Oct 24 13:06:35.938757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 13:06:35.938925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 13:06:35.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 13:06:35.939097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 13:06:35.939176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] command line args: [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 13:06:35.939258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] command line args: [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 13:06:35.939388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 13:06:35.939566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 13:06:35.939648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ================================================================================ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 13:06:35.939715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ================================================================================ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 13:06:35.939820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] admin_token = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.939952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] admin_token = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.940065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_dir = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.940189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_dir = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.940304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.945504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.945726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] config_source = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.945836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_source = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.945916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] debug = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.945989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] debug = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.946061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.946284 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.946420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.946526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.946526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.946711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.946862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.946979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.947084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.947182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.947303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.947387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_config_append = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_config_append = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_dir = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_dir = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.948963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949496 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949857 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.949937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.950926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] publish_errors = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] publish_errors = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.951938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] standard_threads = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] standard_threads = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.952939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_journal = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_journal = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953085 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_json = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_json = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_stderr = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_stderr = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] use_syslog = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_syslog = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 13:06:35.953761 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.953836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.953915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.953992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954420 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.954997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955511 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.955954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.956881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957468 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957686 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.957970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958703 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.958992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959145 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.959942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.960969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961167 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.961989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.962951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963703 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.963956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.964039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.964194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.964328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.964457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.964539 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.964691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.964836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.964955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.965058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.965197 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.965274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.965431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.965517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.965675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.965753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.965916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.966039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.966186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.966266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.966426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.966506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.966678 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.966758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.966923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.967958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.968105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.968186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.968364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.968448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.968613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.968692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.968854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.968934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.969101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.969179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.969645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.969749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.969831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.969911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.970060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.970139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.970297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.970375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.970533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.970622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.970793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.970872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.971036 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.971114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.971283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.971377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.971525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.971610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.971763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.971841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.972005 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.972094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.972242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.972368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.972502 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.972582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.972740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.972831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.973958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.974121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.974209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.974358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.974438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.974605 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.974684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.974844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.974931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.975100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.975288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.975369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.975440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.975591 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.975700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.975822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.external = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.975910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.external = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.976054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.976134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.976374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.976463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.976608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.979925 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.979925 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.password = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.password = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] auth.token = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.token = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.980947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.981997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982284 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982669 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.982966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983110 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.983995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.984978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985041 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985243 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.985974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986036 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986250 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.986941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.987102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.987213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.987366 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.987448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.987618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.987707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.987854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.987934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.988084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.988162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.988333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.988431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.988581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.988666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.988826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.988905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.989072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.989158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.989328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.989409 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.989575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.989654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.989844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.989934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.990081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.990160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.990319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.990399 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.990550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.990638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.990787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.990866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.991975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.992063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.992208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.992328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.992483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.992564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.992732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.992969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.993215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.993463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.993694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.993773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.993934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.994164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.994942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.995031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.995181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.995292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.995426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.995560 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.995667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.996023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.996288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.996552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.996789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.997026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.997277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.997528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.997768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.998009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.998250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.998484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.998727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.998885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.998987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.999272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.999367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.999455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.999579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.999713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:35.999973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.000060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.000234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.000518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.000601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.000888 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.000968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.001153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.001390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.001479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.001643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.001845 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.001926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.002125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.002304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.002395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.002612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.002769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.002849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.003082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.003221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.003309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.003549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.003698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.003813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.004050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.004183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.004322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.004576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.004694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.004826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.005200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.005291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.005469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.005715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.005847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.005973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.006223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.006399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.006480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.006701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.006913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.006992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.007186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.007440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.007602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.007680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.007930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.008105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.008291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.008580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.008726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.008827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.009100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.009295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.009377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.009629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.009827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.009962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.010669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.010913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.010992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.011161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.011266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.011414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.011518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.011666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.011787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.011939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.012042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.012149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.012376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.012462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.012632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.012725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.012884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.012964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.013122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.013201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.013364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.013617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.013698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.013872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.014109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.014189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.014386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.014474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.014649 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.014729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.014898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.014977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.015139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.015228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.015384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.015463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.015687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.015772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.015933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.016021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.016168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.016248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.016433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.016513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.016671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.016762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.016922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.017001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.017170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.017253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.017406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.017490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.017675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.017763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.017923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.018075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.018159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.018314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.018395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.018568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.018649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.018817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.018904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.019052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.019133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.019300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.019386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.019531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.019619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.019780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.020372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.020461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.020536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.020536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.025846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.025965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026736 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026736 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.026939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027311 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.027955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.028988 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.029964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030318 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.030986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031273 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.031936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032151 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.032934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 13:06:36.033007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.033902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.034146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.034429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.034650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.034884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.035127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.035355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.035593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.035823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 13:06:36.035991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 13:06:36.060818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.061046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.061268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.061968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.062198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.062418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.069610 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.070707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.102222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.102450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.102566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.102703 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.161341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.161514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.210948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.211226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.212540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.212775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.389733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.389993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.390179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.390399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.390636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.390835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.398742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.398973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.399196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.399416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.399696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.399923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.415657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.415844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.416073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.416331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.416559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.416782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.417002 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.417340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.417610 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.417850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.418071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.418287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.418502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.418713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.428579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.429379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.445889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.446127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.446359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.446496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.446585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.446732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.446818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.446957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.447044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.447175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.447258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.447966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.448179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.448415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 13:06:36.465096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.465308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 13:06:36.465394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.465556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 13:06:36.465641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.465845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 13:06:36.465930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.466082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 13:06:36.466174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.466356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.466442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.466632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.466715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.466895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.466986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.467175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.476811 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.477095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.477251 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.477373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.477529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.477662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.477796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.478076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.484707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.485436 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.487621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.487910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.488195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.488384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.488506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.488785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.488872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.489049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.489130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.489323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.489415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.489603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.489686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.489880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.489963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.490187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.490271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.490466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.490549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.490734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.491015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.491282 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.513653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.515121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.515206 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.515409 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.516617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.516902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.520082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.520400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.520695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.521699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.521984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.522274 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.527619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.527910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.529622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.529908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.533696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.534007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.534303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.534596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.534886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.535164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.535446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.535733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.535816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.536122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.536441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.536730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.537024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.537304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.537588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.537873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.551439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.551724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.553782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.554059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.555616 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.558589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.559417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.559719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.561872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.562168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.563985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.566495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.567190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.569696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.570439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.570742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.571040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.571334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.571632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.571946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.573562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.573860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.574141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.574436 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.574712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.574999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.588306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.588602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.588875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.589146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.591073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.591361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.591626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.591887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.596353 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.597892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.599175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.599449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.599740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.600839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.602383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.602473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.602656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.602741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.602978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.603266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.603692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.605396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.605751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.606041 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.606321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.606712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.614915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.615202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.615485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.615767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.616170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.616497 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.616783 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.618161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.618443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.618730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.619008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.619906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.620197 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.622014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.633513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.637073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.638343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.640125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.641461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.641771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.643319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.643608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.644006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.644529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.646059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.647772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.648056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.648531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.650661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.652147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.656011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.656347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.656436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.656719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.657005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.657887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.660869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.661266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.661551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.661825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.665074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.665372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.665709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.669644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.669939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.670219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.672740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.673031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.673320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.673601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.673876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.674152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.674429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.674720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.675003 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.675449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.675901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.676178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.676536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.676838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.677154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 13:06:36.677439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.677722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.677995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.678265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.678532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.678806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.679094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.679378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.679647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.679916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.680177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.680474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.680779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 13:06:36.711891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e8118639c0 pid: 81603 (default app) Oct 24 13:06:36.715257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4cd0-356b-47c5-ba91-11c7a84f6cb9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:36.715518 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e8118639c0 pid: 81604 (default app) Oct 24 13:06:36.715685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4cd0-356b-47c5-ba91-11c7a84f6cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:36.715805 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa4cd0-356b-47c5-ba91-11c7a84f6cb9 None None] PATH_INFO: `/v3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:36.720240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Mon Oct 24 13:06:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:44.180681 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbe520-4a95-4ca1-833c-f4a4d5fbb5a4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:44.181173 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbe520-4a95-4ca1-833c-f4a4d5fbb5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:44.181258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbe520-4a95-4ca1-833c-f4a4d5fbb5a4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:44.185998 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:06:44.191275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:44.191680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:44.191831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:44.262087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: INFO dbcounter [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] Registered counter for database keystone Oct 24 13:06:44.280576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG oslo_db.sqlalchemy.engines [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 13:06:44.286692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writer thread running {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 13:06:44.463514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG passlib.handlers.bcrypt [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81603) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 24 13:06:44.463922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG passlib.handlers.bcrypt [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81603) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 24 13:06:44.522055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:44.523665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:44.650017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 13:06:45.065307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbe520-4a95-4ca1-833c-f4a4d5fbb5a4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:45.133475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: INFO dbcounter [None req-5abbe520-4a95-4ca1-833c-f4a4d5fbb5a4 None None] Registered counter for database keystone Oct 24 13:06:45.154983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG oslo_db.sqlalchemy.engines [None req-5abbe520-4a95-4ca1-833c-f4a4d5fbb5a4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 13:06:45.161055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writer thread running {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 13:06:45.276544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbe520-4a95-4ca1-833c-f4a4d5fbb5a4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:45.280794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67084da6-a6bd-49d3-9b0a-0609ef7acb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:45.281405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084da6-a6bd-49d3-9b0a-0609ef7acb5a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:45.281679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084da6-a6bd-49d3-9b0a-0609ef7acb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:45.281945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084da6-a6bd-49d3-9b0a-0609ef7acb5a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:45.289986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67084da6-a6bd-49d3-9b0a-0609ef7acb5a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:45.341888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING keystone.common.rbac_enforcer.enforcer [None req-67084da6-a6bd-49d3-9b0a-0609ef7acb5a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 24 13:06:45.342175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67084da6-a6bd-49d3-9b0a-0609ef7acb5a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:45.351484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Oct 24 13:06:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:06:45.357585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:45.374106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ede0e9-11aa-430c-923a-f2f2437cd5f4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:45.378477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5e000-c210-4acf-b3f7-b427120615b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:45.379571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e000-c210-4acf-b3f7-b427120615b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:45.379854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e000-c210-4acf-b3f7-b427120615b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:45.380135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e000-c210-4acf-b3f7-b427120615b4 admin admin] PATH_INFO: `/v3/projects/c5f7bf50f5a94c7694720e6be18c9de2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:45.381712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5e000-c210-4acf-b3f7-b427120615b4 admin admin] RBAC: Authorizing `identity:get_project(project_id=c5f7bf50f5a94c7694720e6be18c9de2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:45.433889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d9a5e000-c210-4acf-b3f7-b427120615b4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 24 13:06:45.434233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5e000-c210-4acf-b3f7-b427120615b4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:45.438702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Oct 24 13:06:45 2022] GET /identity/v3/projects/c5f7bf50f5a94c7694720e6be18c9de2 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:06:47.548051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992e914-3054-439b-987c-4b61dbc98ba6 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:47.548598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992e914-3054-439b-987c-4b61dbc98ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:47.548598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992e914-3054-439b-987c-4b61dbc98ba6 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:47.552677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:06:47.558719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c9300-7f5a-4edd-b580-0fd667888508 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:47.559122 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c9300-7f5a-4edd-b580-0fd667888508 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:47.559259 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c9300-7f5a-4edd-b580-0fd667888508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:47.578891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e55c9300-7f5a-4edd-b580-0fd667888508 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:47.580411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e55c9300-7f5a-4edd-b580-0fd667888508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:47.588069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:06:47.857672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992e914-3054-439b-987c-4b61dbc98ba6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:47.869608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992e914-3054-439b-987c-4b61dbc98ba6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:47.873802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e3c5e-9bae-4346-92cf-8871aff1ba73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:47.874386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e3c5e-9bae-4346-92cf-8871aff1ba73 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:47.874665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e3c5e-9bae-4346-92cf-8871aff1ba73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:47.874937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e3c5e-9bae-4346-92cf-8871aff1ba73 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:47.882486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e3c5e-9bae-4346-92cf-8871aff1ba73 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:47.884250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e3c5e-9bae-4346-92cf-8871aff1ba73 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:47.892916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Oct 24 13:06:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:06:47.898055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c9300-7f5a-4edd-b580-0fd667888508 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:47.907534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c9300-7f5a-4edd-b580-0fd667888508 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:47.911782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2334d2b-9057-49c8-9c0a-ff8cbc8f6ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:47.912435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334d2b-9057-49c8-9c0a-ff8cbc8f6ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:47.912718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334d2b-9057-49c8-9c0a-ff8cbc8f6ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:47.912992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2334d2b-9057-49c8-9c0a-ff8cbc8f6ed4 admin admin] PATH_INFO: `/v3/users/742bb1241cf54598b570a2fd87179834` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:47.914277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2334d2b-9057-49c8-9c0a-ff8cbc8f6ed4 admin admin] RBAC: Authorizing `identity:get_user(user_id=742bb1241cf54598b570a2fd87179834)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:47.916080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2334d2b-9057-49c8-9c0a-ff8cbc8f6ed4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:47.918244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Mon Oct 24 13:06:47 2022] GET /identity/v3/users/742bb1241cf54598b570a2fd87179834 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:50.077945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4a2cc-1b7b-477b-9df8-15df3076e3ee None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.078463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4a2cc-1b7b-477b-9df8-15df3076e3ee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.078463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4a2cc-1b7b-477b-9df8-15df3076e3ee None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.079790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:50.085111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899640-a4b3-4f3d-b0e4-c30477a84cda None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.085504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899640-a4b3-4f3d-b0e4-c30477a84cda None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.085663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21899640-a4b3-4f3d-b0e4-c30477a84cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.103653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-21899640-a4b3-4f3d-b0e4-c30477a84cda None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:50.104863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-21899640-a4b3-4f3d-b0e4-c30477a84cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:50.138811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118805e-233c-436b-a680-30776b841e3f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.139092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118805e-233c-436b-a680-30776b841e3f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.139374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118805e-233c-436b-a680-30776b841e3f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.140724 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:06:50.145703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.145988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.146266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.185597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:06:50.188200 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG passlib.handlers.bcrypt [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81604) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 24 13:06:50.188619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG passlib.handlers.bcrypt [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81604) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 24 13:06:50.237453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:50.239004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:50.335948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:06:50.454635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21899640-a4b3-4f3d-b0e4-c30477a84cda None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:50.479995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:50.558288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21899640-a4b3-4f3d-b0e4-c30477a84cda None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:50.591090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7419aff-29e1-4410-913b-f144fe83d19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:50.591654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7419aff-29e1-4410-913b-f144fe83d19c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.591933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7419aff-29e1-4410-913b-f144fe83d19c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.592204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7419aff-29e1-4410-913b-f144fe83d19c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.603722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7419aff-29e1-4410-913b-f144fe83d19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:06:50.606153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a7419aff-29e1-4410-913b-f144fe83d19c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:06:50.612659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:06:50.613685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:06:50.614566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:06:50.621573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7419aff-29e1-4410-913b-f144fe83d19c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:50.700974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:50.708146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81290c9f-fc92-40f0-90f0-eedca43e4a63 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:50.710410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:50.718780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7419aff-29e1-4410-913b-f144fe83d19c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:50.744762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:50.746220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.746505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.746770 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.747709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:50.749148 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:50.753971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:50.754569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.754848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.755118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.755770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:06:50.756073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:50.757477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:50.769317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:06:50.774938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:50.823538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.sql.core [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:06:50.823538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:06:50.823538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [parameters: {'id': '25a0b7d27b4f49c98b18cdec48a9da64', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:06:50.823538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:06:50.842182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 13:06:50.843096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application yield resource Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise self.value Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application session.commit() Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:06:50.844535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:06:50.845646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:06:50.846812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:06:50.847732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [parameters: {'id': '25a0b7d27b4f49c98b18cdec48a9da64', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 13:06:50.849039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:06:50.849931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 8/16] 10.222.0.48 () {60 vars in 1137 bytes} [Mon Oct 24 13:06:50 2022] POST /identity/v3/domains => generated 190 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:06:50.851076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:50.873598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb34f49-6db6-44a5-bd86-8f5ef2944e0e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:50.907309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0779f-3070-4914-bedf-89635a124ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:50.907937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0779f-3070-4914-bedf-89635a124ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.908211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0779f-3070-4914-bedf-89635a124ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.908519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0779f-3070-4914-bedf-89635a124ab4 None admin] PATH_INFO: `/v3/users/742bb1241cf54598b570a2fd87179834` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.909635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0779f-3070-4914-bedf-89635a124ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=742bb1241cf54598b570a2fd87179834)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:50.911080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0779f-3070-4914-bedf-89635a124ab4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:50.912655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:06:50 2022] GET /identity/v3/users/742bb1241cf54598b570a2fd87179834 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:06:50.918206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0779f-3070-4914-bedf-89635a124ab4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:50.932074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:50.950191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a61fd-1648-47c7-8ee0-cdfb9a0ccae8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:50.983460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:50.984315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.984593 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.984864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:51.002952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:51.004424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:51.015157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0779f-3070-4914-bedf-89635a124ab4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:06:51.020184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:06:51.021211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:06:51.022431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:06:51.022431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:06:51.022431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:51.022431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 9/18] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:06:51.027028 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:51.047861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:51.050425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8dd40-9a0f-4157-ab0e-95c74c722dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:51.051052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8dd40-9a0f-4157-ab0e-95c74c722dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:51.051339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8dd40-9a0f-4157-ab0e-95c74c722dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:51.051611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8dd40-9a0f-4157-ab0e-95c74c722dc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:51.052862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8dd40-9a0f-4157-ab0e-95c74c722dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:51.054304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8dd40-9a0f-4157-ab0e-95c74c722dc5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:51.055899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 10/19] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:06:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:06:51.061840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8dd40-9a0f-4157-ab0e-95c74c722dc5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:51.132972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5ecea-6999-4fc6-85d7-9a83db59fe07 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:51.140530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:51.158184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8dd40-9a0f-4157-ab0e-95c74c722dc5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:51.166457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03f0e5-6014-4bd6-8f82-5c9099767d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:51.167628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f0e5-6014-4bd6-8f82-5c9099767d0d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:51.167905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f0e5-6014-4bd6-8f82-5c9099767d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:51.168178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03f0e5-6014-4bd6-8f82-5c9099767d0d None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:51.168852 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03f0e5-6014-4bd6-8f82-5c9099767d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:06:51.169151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03f0e5-6014-4bd6-8f82-5c9099767d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:51.170525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03f0e5-6014-4bd6-8f82-5c9099767d0d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:51.187782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:06:51.191835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1772cb-60a5-4169-8291-96407e0954a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:51.192618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772cb-60a5-4169-8291-96407e0954a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:51.192964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772cb-60a5-4169-8291-96407e0954a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:51.193305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1772cb-60a5-4169-8291-96407e0954a8 None admin] PATH_INFO: `/v3/domains/default/users/742bb1241cf54598b570a2fd87179834/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:51.195362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1772cb-60a5-4169-8291-96407e0954a8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=742bb1241cf54598b570a2fd87179834, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:51.196900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1772cb-60a5-4169-8291-96407e0954a8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:51.398136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 11/21] 10.222.0.48 () {58 vars in 1361 bytes} [Mon Oct 24 13:06:51 2022] PUT /identity/v3/domains/default/users/742bb1241cf54598b570a2fd87179834/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:06:53.731553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4141-40b6-44a4-9dc6-242bf8bdca8e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:53.732123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4141-40b6-44a4-9dc6-242bf8bdca8e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:53.732123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b4141-40b6-44a4-9dc6-242bf8bdca8e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:53.733439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:53.739508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5281-f75c-4460-9097-bb979254ba17 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:53.739954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5281-f75c-4460-9097-bb979254ba17 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:53.740134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a5281-f75c-4460-9097-bb979254ba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:53.765578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-193a5281-f75c-4460-9097-bb979254ba17 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:53.767205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-193a5281-f75c-4460-9097-bb979254ba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:53.864041 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 12/23] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:06:53.903351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c19ff-af68-4a27-b4e3-7d6493edf22c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:53.903968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c19ff-af68-4a27-b4e3-7d6493edf22c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:53.904477 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c19ff-af68-4a27-b4e3-7d6493edf22c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:53.906265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 12/24] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:53.911948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0c976-f982-4a86-91ed-e790f0b7009e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:53.912244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0c976-f982-4a86-91ed-e790f0b7009e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:53.912541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0c976-f982-4a86-91ed-e790f0b7009e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:53.931476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-75a0c976-f982-4a86-91ed-e790f0b7009e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:53.933242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-75a0c976-f982-4a86-91ed-e790f0b7009e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:54.020406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:54.029445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 13/25] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:06:54.304114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e90781-8e48-49ee-b540-5a0935e12676 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:54.304799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e90781-8e48-49ee-b540-5a0935e12676 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:54.305294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e90781-8e48-49ee-b540-5a0935e12676 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:54.307319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 13/26] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:54.325212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700acfe-ad8c-4b90-ba0a-1c5a5700dffe None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:54.325837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700acfe-ad8c-4b90-ba0a-1c5a5700dffe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:54.326317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700acfe-ad8c-4b90-ba0a-1c5a5700dffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:54.351039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4700acfe-ad8c-4b90-ba0a-1c5a5700dffe None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:54.351492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e90781-8e48-49ee-b540-5a0935e12676 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:54.353414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4700acfe-ad8c-4b90-ba0a-1c5a5700dffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:54.436450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 14/27] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:06:54.443885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a94d-b34c-4f5a-a2af-76dcae1cca46 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:54.444177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a94d-b34c-4f5a-a2af-76dcae1cca46 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:54.444491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a94d-b34c-4f5a-a2af-76dcae1cca46 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:54.448693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 15/28] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:06:54.465678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8bed1-93e7-47af-8b66-48780fb2d944 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:54.465965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8bed1-93e7-47af-8b66-48780fb2d944 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:54.466244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8bed1-93e7-47af-8b66-48780fb2d944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:54.476622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:54.493729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e90781-8e48-49ee-b540-5a0935e12676 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:54.505938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f1a8bed1-93e7-47af-8b66-48780fb2d944 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:54.507157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f1a8bed1-93e7-47af-8b66-48780fb2d944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:54.561732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:54.562276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:54.562554 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:54.562824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:54.582928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:54.586225 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:06:54.611964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:06:54.612911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:06:54.613783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:06:54.613783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:06:54.613783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:54.613783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 14/29] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:06:54.614004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:54.635012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 16/30] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:06:54.635741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8bed1-93e7-47af-8b66-48780fb2d944 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:54.743663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:54.806977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8bed1-93e7-47af-8b66-48780fb2d944 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:54.809328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fe6fd-67f9-4647-aa65-7b12deb25a27 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:54.852770 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:54.860241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:54.861408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda6449-8eac-44b1-bf31-6192e19bc936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:54.866082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda6449-8eac-44b1-bf31-6192e19bc936 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:54.866082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda6449-8eac-44b1-bf31-6192e19bc936 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:54.866082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda6449-8eac-44b1-bf31-6192e19bc936 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:54.866312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:54.866312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:54.866312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:54.866312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:06:54.866312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:54.866779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:54.888507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 17/31] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:54.889965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:54.900889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda6449-8eac-44b1-bf31-6192e19bc936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5dda6449-8eac-44b1-bf31-6192e19bc936 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:06:54.908824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:06:54.909698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:06:54.910515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:06:54.920751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 15/32] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:06:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:06:54.921629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda6449-8eac-44b1-bf31-6192e19bc936 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:54.994042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc449f-f898-4fa0-994b-299d7faab889 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:55.015193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:55.030492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948d598-d2d5-4739-aafb-94067976cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:55.036623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d598-d2d5-4739-aafb-94067976cd60 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.036623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d598-d2d5-4739-aafb-94067976cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.036623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d598-d2d5-4739-aafb-94067976cd60 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.036623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948d598-d2d5-4739-aafb-94067976cd60 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:55.036623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948d598-d2d5-4739-aafb-94067976cd60 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:55.062665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda6449-8eac-44b1-bf31-6192e19bc936 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:55.105438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac612e4-2d07-4046-ab5d-6ca40a9049e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:55.105997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac612e4-2d07-4046-ab5d-6ca40a9049e2 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.106375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac612e4-2d07-4046-ab5d-6ca40a9049e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.106654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac612e4-2d07-4046-ab5d-6ca40a9049e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.107336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac612e4-2d07-4046-ab5d-6ca40a9049e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:55.108883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac612e4-2d07-4046-ab5d-6ca40a9049e2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:55.140174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 18/33] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Oct 24 13:06:54 2022] POST /identity/v3/roles => generated 219 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:06:55.141837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948d598-d2d5-4739-aafb-94067976cd60 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:55.293711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948d598-d2d5-4739-aafb-94067976cd60 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:55.333011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:55.333966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.334420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.334844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.335676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:06:55.336204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:55.337815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:55.348696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 19/34] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:06:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:55.349797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:55.385395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 16/35] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:06:54 2022] POST /identity/v3/projects => generated 301 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:06:55.386745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac612e4-2d07-4046-ab5d-6ca40a9049e2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:55.405373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:55.475687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6006a-ee1a-4ab9-abad-618accbc5769 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:55.500679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:55.520074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d5bd9-76a6-48ec-b29e-8497d5bdf2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:55.520758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d5bd9-76a6-48ec-b29e-8497d5bdf2c7 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.521044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d5bd9-76a6-48ec-b29e-8497d5bdf2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.521319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d5bd9-76a6-48ec-b29e-8497d5bdf2c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.521997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d5bd9-76a6-48ec-b29e-8497d5bdf2c7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:55.523671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d5bd9-76a6-48ec-b29e-8497d5bdf2c7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:55.569870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac612e4-2d07-4046-ab5d-6ca40a9049e2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:55.592858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 20/36] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Oct 24 13:06:55 2022] POST /identity/v3/roles => generated 225 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:06:55.597973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dc4e7-f78a-428f-84ad-e53bc9d0550a None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.598331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dc4e7-f78a-428f-84ad-e53bc9d0550a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.598536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dc4e7-f78a-428f-84ad-e53bc9d0550a None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.600300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 21/37] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:55.604889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16846e-ed43-4ac3-85f0-a9f984164d12 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.604889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16846e-ed43-4ac3-85f0-a9f984164d12 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.604889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16846e-ed43-4ac3-85f0-a9f984164d12 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.604889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 22/38] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:55.608895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c39b8-ddb0-4c26-a030-166d0cdb0744 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.609210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c39b8-ddb0-4c26-a030-166d0cdb0744 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.609456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c39b8-ddb0-4c26-a030-166d0cdb0744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.622230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfafbc-1dcd-4cee-8045-d332e7a01451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:55.623263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfafbc-1dcd-4cee-8045-d332e7a01451 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.623748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfafbc-1dcd-4cee-8045-d332e7a01451 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.624227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfafbc-1dcd-4cee-8045-d332e7a01451 None admin] PATH_INFO: `/v3/users/742bb1241cf54598b570a2fd87179834` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.626227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfafbc-1dcd-4cee-8045-d332e7a01451 None admin] RBAC: Authorizing `identity:get_user(user_id=742bb1241cf54598b570a2fd87179834)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:55.628198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfafbc-1dcd-4cee-8045-d332e7a01451 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:55.630617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 17/39] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:06:55 2022] GET /identity/v3/users/742bb1241cf54598b570a2fd87179834 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:55.632436 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf782992-ad17-4b67-b201-23c7162505d8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.632959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf782992-ad17-4b67-b201-23c7162505d8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.633459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf782992-ad17-4b67-b201-23c7162505d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.660312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cf782992-ad17-4b67-b201-23c7162505d8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:55.661973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cf782992-ad17-4b67-b201-23c7162505d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:55.745785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b07c39b8-ddb0-4c26-a030-166d0cdb0744 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:55.747003 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b07c39b8-ddb0-4c26-a030-166d0cdb0744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:55.753258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 18/40] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:06:55.753963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf782992-ad17-4b67-b201-23c7162505d8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:55.810320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:55.835254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 23/41] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:06:55.881256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf782992-ad17-4b67-b201-23c7162505d8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:55.915800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2849a0-d6ac-4b96-b543-dfe24af4d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:55.916400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2849a0-d6ac-4b96-b543-dfe24af4d5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:55.917221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2849a0-d6ac-4b96-b543-dfe24af4d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:55.917499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2849a0-d6ac-4b96-b543-dfe24af4d5d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:55.918618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2849a0-d6ac-4b96-b543-dfe24af4d5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:55.920073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2849a0-d6ac-4b96-b543-dfe24af4d5d9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:55.921855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 19/42] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:06:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:55.927420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c39b8-ddb0-4c26-a030-166d0cdb0744 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:55.959051 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:56.022804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2849a0-d6ac-4b96-b543-dfe24af4d5d9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:56.029642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c39b8-ddb0-4c26-a030-166d0cdb0744 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:56.065258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:56.065861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:56.066141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:56.066412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:56.067128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.domain.id=default, user.id=742bb1241cf54598b570a2fd87179834) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:06:56.067433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:56.068893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:56.076226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 24/43] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Oct 24 13:06:55 2022] GET /identity/v3/role_assignments?user.id=742bb1241cf54598b570a2fd87179834&scope.domain.id=default&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:56.107443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:56.140522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2849a0-d6ac-4b96-b543-dfe24af4d5d9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:56.153030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:56.179773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003539e-d210-44ae-b45a-5cbd8fc43fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:56.180351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003539e-d210-44ae-b45a-5cbd8fc43fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:56.180638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003539e-d210-44ae-b45a-5cbd8fc43fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:56.180916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003539e-d210-44ae-b45a-5cbd8fc43fb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:56.182063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003539e-d210-44ae-b45a-5cbd8fc43fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:56.183577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003539e-d210-44ae-b45a-5cbd8fc43fb9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:56.185229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 20/44] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:06:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:56.190909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003539e-d210-44ae-b45a-5cbd8fc43fb9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:56.220705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:56.226683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaedd7-9245-4b00-ad00-b9235cd681d7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:56.261921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ca2f7-4677-440e-844f-ee08d08be21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:56.262527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ca2f7-4677-440e-844f-ee08d08be21e None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:56.263514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ca2f7-4677-440e-844f-ee08d08be21e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:56.263514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ca2f7-4677-440e-844f-ee08d08be21e None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:56.263739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ca2f7-4677-440e-844f-ee08d08be21e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:56.265140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ca2f7-4677-440e-844f-ee08d08be21e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:56.301065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003539e-d210-44ae-b45a-5cbd8fc43fb9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:56.334072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a9288-c30a-4fc3-bf9f-7564ec543490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:56.334671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a9288-c30a-4fc3-bf9f-7564ec543490 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:56.334953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a9288-c30a-4fc3-bf9f-7564ec543490 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:56.335229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a9288-c30a-4fc3-bf9f-7564ec543490 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:56.335878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a9288-c30a-4fc3-bf9f-7564ec543490 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:56.337345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a9288-c30a-4fc3-bf9f-7564ec543490 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:56.361757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 25/45] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Oct 24 13:06:56 2022] POST /identity/v3/roles => generated 223 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:06:56.420957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 21/46] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:06:56 2022] POST /identity/v3/projects => generated 312 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:06:58.560630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d545c4-7eb5-4ae2-aca1-fed5c59e5eb4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:58.561148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d545c4-7eb5-4ae2-aca1-fed5c59e5eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:58.561230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d545c4-7eb5-4ae2-aca1-fed5c59e5eb4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:58.562529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 26/47] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:58.567704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30aca7-32e7-4508-a566-6fc2ee6e0bb9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:58.568120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30aca7-32e7-4508-a566-6fc2ee6e0bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:58.568262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30aca7-32e7-4508-a566-6fc2ee6e0bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:58.586783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5b30aca7-32e7-4508-a566-6fc2ee6e0bb9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:58.587952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b30aca7-32e7-4508-a566-6fc2ee6e0bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:58.611174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:58.653162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 22/48] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:06:58.920778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d545c4-7eb5-4ae2-aca1-fed5c59e5eb4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:59.019112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d545c4-7eb5-4ae2-aca1-fed5c59e5eb4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:59.050928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d619ba0-263c-4961-9cc9-2d62f7c35eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:59.051534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d619ba0-263c-4961-9cc9-2d62f7c35eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:59.051814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d619ba0-263c-4961-9cc9-2d62f7c35eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:59.052085 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d619ba0-263c-4961-9cc9-2d62f7c35eb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:59.053198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d619ba0-263c-4961-9cc9-2d62f7c35eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:59.054634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d619ba0-263c-4961-9cc9-2d62f7c35eb6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:59.056149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 27/49] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:06:59.062116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30aca7-32e7-4508-a566-6fc2ee6e0bb9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:59.161092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30aca7-32e7-4508-a566-6fc2ee6e0bb9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:59.193921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c816d-b635-41d0-8c7c-ff74dac6551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:59.194468 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c816d-b635-41d0-8c7c-ff74dac6551a None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:59.194757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c816d-b635-41d0-8c7c-ff74dac6551a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:59.195034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c816d-b635-41d0-8c7c-ff74dac6551a None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:59.195687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c816d-b635-41d0-8c7c-ff74dac6551a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:59.197157 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c816d-b635-41d0-8c7c-ff74dac6551a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:59.303758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 23/50] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Oct 24 13:06:59 2022] POST /identity/v3/projects => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:01.430158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8441-d23b-48c1-8293-54efd0a31762 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:01.430654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8441-d23b-48c1-8293-54efd0a31762 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:01.430736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8441-d23b-48c1-8293-54efd0a31762 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:01.432058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 28/51] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:01.437375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76d0a4-7fc0-4d07-82cd-aa82e620ca72 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:01.437747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76d0a4-7fc0-4d07-82cd-aa82e620ca72 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:01.437940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76d0a4-7fc0-4d07-82cd-aa82e620ca72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:01.456583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2d76d0a4-7fc0-4d07-82cd-aa82e620ca72 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:01.457762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2d76d0a4-7fc0-4d07-82cd-aa82e620ca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:01.465659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:01.626505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 24/52] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:01.899136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee8441-d23b-48c1-8293-54efd0a31762 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:01.924887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:01.998606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee8441-d23b-48c1-8293-54efd0a31762 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:02.031492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94447776-eaea-412d-ab08-603146b3d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:02.032103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94447776-eaea-412d-ab08-603146b3d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:02.032412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94447776-eaea-412d-ab08-603146b3d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:02.032687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94447776-eaea-412d-ab08-603146b3d3b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:02.033754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94447776-eaea-412d-ab08-603146b3d3b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:02.035184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94447776-eaea-412d-ab08-603146b3d3b1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:02.036736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 29/53] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:02.042204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76d0a4-7fc0-4d07-82cd-aa82e620ca72 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:02.138735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76d0a4-7fc0-4d07-82cd-aa82e620ca72 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:02.170934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9971e4-907f-4238-ac86-fe85365a4c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:02.171481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9971e4-907f-4238-ac86-fe85365a4c64 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:02.171774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9971e4-907f-4238-ac86-fe85365a4c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:02.172050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9971e4-907f-4238-ac86-fe85365a4c64 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:02.172730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9971e4-907f-4238-ac86-fe85365a4c64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:02.174159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9971e4-907f-4238-ac86-fe85365a4c64 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:02.269574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:02.274528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 25/54] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:02 2022] POST /identity/v3/users => generated 275 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:05.401218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e6f5c-4d23-4aa5-9f5e-409b35e3f280 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:05.401710 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e6f5c-4d23-4aa5-9f5e-409b35e3f280 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:05.401802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e6f5c-4d23-4aa5-9f5e-409b35e3f280 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:05.403443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 30/55] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:05.408461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351262db-1519-463e-8484-68b3b3e0b10d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:05.408886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351262db-1519-463e-8484-68b3b3e0b10d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:05.409052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351262db-1519-463e-8484-68b3b3e0b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:05.433237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-351262db-1519-463e-8484-68b3b3e0b10d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:05.434438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-351262db-1519-463e-8484-68b3b3e0b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:05.525323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 26/56] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:05.797636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026463f6-7437-44a6-ae11-14122354e05b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:05.798301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026463f6-7437-44a6-ae11-14122354e05b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:05.798807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026463f6-7437-44a6-ae11-14122354e05b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:05.800395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 31/57] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:05.820141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0121-b841-492d-bfad-c24b4b5e0632 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:05.820769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0121-b841-492d-bfad-c24b4b5e0632 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:05.821270 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0121-b841-492d-bfad-c24b4b5e0632 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:05.823420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 32/58] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:07:05.829638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b013-bb15-4326-8e3b-332500384562 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:05.830211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b013-bb15-4326-8e3b-332500384562 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:05.830683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b013-bb15-4326-8e3b-332500384562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:05.832906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274dde4-38b8-49b1-925e-668009ac1308 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:05.833415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274dde4-38b8-49b1-925e-668009ac1308 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:05.835118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274dde4-38b8-49b1-925e-668009ac1308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:05.867978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5b56b013-bb15-4326-8e3b-332500384562 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:05.870087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5b56b013-bb15-4326-8e3b-332500384562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:05.871224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9274dde4-38b8-49b1-925e-668009ac1308 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:05.874667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9274dde4-38b8-49b1-925e-668009ac1308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:06.011242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 33/59] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:06.017566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 27/60] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:06.018695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b013-bb15-4326-8e3b-332500384562 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.067524 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:06.085908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96792-bb9d-4772-9e99-7fd7a2b1146e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.086194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96792-bb9d-4772-9e99-7fd7a2b1146e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.086656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96792-bb9d-4772-9e99-7fd7a2b1146e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.088192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 28/61] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:07:06.094121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646d256-5be6-4017-b704-5fec6235c1dd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.094414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646d256-5be6-4017-b704-5fec6235c1dd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.094699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646d256-5be6-4017-b704-5fec6235c1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.145395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4646d256-5be6-4017-b704-5fec6235c1dd None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:06.147031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4646d256-5be6-4017-b704-5fec6235c1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:06.189157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b013-bb15-4326-8e3b-332500384562 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:06.230704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd1aa-4aec-47c6-be53-603e974079cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:06.231339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd1aa-4aec-47c6-be53-603e974079cb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.231620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd1aa-4aec-47c6-be53-603e974079cb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.231894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dd1aa-4aec-47c6-be53-603e974079cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.245845 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:06.250696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8dd1aa-4aec-47c6-be53-603e974079cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1e8dd1aa-4aec-47c6-be53-603e974079cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:06.256681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:06.261450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:06.262230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 34/62] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:06.264828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd1aa-4aec-47c6-be53-603e974079cb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.275374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 29/63] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:06.333612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646d256-5be6-4017-b704-5fec6235c1dd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.381862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:06.398836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dd1aa-4aec-47c6-be53-603e974079cb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:06.444936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:06.446587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.447012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.447428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.448319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:06.448778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:06.451101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:06.462300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 35/64] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:06.462891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646d256-5be6-4017-b704-5fec6235c1dd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:06.463369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.500878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd3552-8665-4032-93f8-f59f7c90a688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:06.501509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3552-8665-4032-93f8-f59f7c90a688 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.501971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3552-8665-4032-93f8-f59f7c90a688 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.502244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd3552-8665-4032-93f8-f59f7c90a688 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.512180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd3552-8665-4032-93f8-f59f7c90a688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-62fd3552-8665-4032-93f8-f59f7c90a688 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:06.516881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:06.517735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:06.518537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 30/65] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:06.519473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1cb7-6a7b-4b76-bc23-6a30b7ad013e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.519750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1cb7-6a7b-4b76-bc23-6a30b7ad013e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.524731 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1cb7-6a7b-4b76-bc23-6a30b7ad013e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.528168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 31/66] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:06.528168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b1cb7-6a7b-4b76-bc23-6a30b7ad013e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.568019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6737a-6aeb-456b-9c3e-baf55744ac78 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:06.583454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:06.601433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245c0d1-b219-445f-8f0a-70c280de0016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:06.601991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c0d1-b219-445f-8f0a-70c280de0016 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.602273 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c0d1-b219-445f-8f0a-70c280de0016 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.602545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245c0d1-b219-445f-8f0a-70c280de0016 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.613144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245c0d1-b219-445f-8f0a-70c280de0016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8245c0d1-b219-445f-8f0a-70c280de0016 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:06.621191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:06.622067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 36/67] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:06.622856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245c0d1-b219-445f-8f0a-70c280de0016 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.625198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:06.636788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b1cb7-6a7b-4b76-bc23-6a30b7ad013e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:06.670916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcbf35-68b1-4d5a-9ade-0f2e2c473db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:06.671538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcbf35-68b1-4d5a-9ade-0f2e2c473db9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.671819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcbf35-68b1-4d5a-9ade-0f2e2c473db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.672099 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcbf35-68b1-4d5a-9ade-0f2e2c473db9 None admin] PATH_INFO: `/v3/users/94f198dd02e44352b78185bbf6b74125` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.686930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcbf35-68b1-4d5a-9ade-0f2e2c473db9 None admin] RBAC: Authorizing `identity:get_user(user_id=94f198dd02e44352b78185bbf6b74125)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:06.688592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcbf35-68b1-4d5a-9ade-0f2e2c473db9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:06.690952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 32/68] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/users/94f198dd02e44352b78185bbf6b74125 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:06.695795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568ddd-bf07-4532-8ce2-6ea609d2bfe3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.695795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568ddd-bf07-4532-8ce2-6ea609d2bfe3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.695795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568ddd-bf07-4532-8ce2-6ea609d2bfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.711875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-57568ddd-bf07-4532-8ce2-6ea609d2bfe3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:06.713623 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-57568ddd-bf07-4532-8ce2-6ea609d2bfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:06.719835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245c0d1-b219-445f-8f0a-70c280de0016 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:06.752864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:06.753460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.753747 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.754032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.754684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:06.754989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:06.756390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:06.764639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 37/69] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:06.765263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.782933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 33/70] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:06.783765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568ddd-bf07-4532-8ce2-6ea609d2bfe3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.818277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:06.831282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:06.866743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b51e12-a1aa-449e-98d2-c2905fd65d14 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:06.891949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568ddd-bf07-4532-8ce2-6ea609d2bfe3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:06.902634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9dcb-bc24-4b54-923d-79c7d8b62e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:06.903262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9dcb-bc24-4b54-923d-79c7d8b62e99 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.903546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9dcb-bc24-4b54-923d-79c7d8b62e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.904025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9dcb-bc24-4b54-923d-79c7d8b62e99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.913935 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d9dcb-bc24-4b54-923d-79c7d8b62e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-922d9dcb-bc24-4b54-923d-79c7d8b62e99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:06.920357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:06.921229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:06.922024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 38/71] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:06.922675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9dcb-bc24-4b54-923d-79c7d8b62e99 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:06.934640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:06.935322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:06.935677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:06.935963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:06.936650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:06.936962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:06.938345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:06.955179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 34/72] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:06.955179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.030713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9dcb-bc24-4b54-923d-79c7d8b62e99 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.055017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:07.056582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:07.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c244c-7022-48e1-8eac-b1b08e900f28 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.065611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f319d3-1cf9-425e-82dd-49e1e45f3fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.065908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f319d3-1cf9-425e-82dd-49e1e45f3fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.069330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f319d3-1cf9-425e-82dd-49e1e45f3fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.069330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f319d3-1cf9-425e-82dd-49e1e45f3fe7 None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.069330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f319d3-1cf9-425e-82dd-49e1e45f3fe7 None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.069330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f319d3-1cf9-425e-82dd-49e1e45f3fe7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.070641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 39/73] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:07.071393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f319d3-1cf9-425e-82dd-49e1e45f3fe7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.097948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e50e8-5eb8-4d0c-9b11-afeb0f00f1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.098595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e50e8-5eb8-4d0c-9b11-afeb0f00f1ba None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.098883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e50e8-5eb8-4d0c-9b11-afeb0f00f1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.099171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e50e8-5eb8-4d0c-9b11-afeb0f00f1ba None admin] PATH_INFO: `/v3/users/94f198dd02e44352b78185bbf6b74125` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.100424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e50e8-5eb8-4d0c-9b11-afeb0f00f1ba None admin] RBAC: Authorizing `identity:get_user(user_id=94f198dd02e44352b78185bbf6b74125)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.101881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e50e8-5eb8-4d0c-9b11-afeb0f00f1ba None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.103670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 35/74] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:06 2022] GET /identity/v3/users/94f198dd02e44352b78185bbf6b74125 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:07.104327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e50e8-5eb8-4d0c-9b11-afeb0f00f1ba None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.170647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f319d3-1cf9-425e-82dd-49e1e45f3fe7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.203171 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.203774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.204055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.204144 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e50e8-5eb8-4d0c-9b11-afeb0f00f1ba None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.204362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.205015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:07.205319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.206708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.215055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 40/75] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:07.215837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.239467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4b24b-9b33-49fb-868d-a0600a047dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.240091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4b24b-9b33-49fb-868d-a0600a047dec None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.240403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4b24b-9b33-49fb-868d-a0600a047dec None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.240683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4b24b-9b33-49fb-868d-a0600a047dec None admin] PATH_INFO: `/v3/users/742bb1241cf54598b570a2fd87179834` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.241800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4b24b-9b33-49fb-868d-a0600a047dec None admin] RBAC: Authorizing `identity:get_user(user_id=742bb1241cf54598b570a2fd87179834)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.243240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4b24b-9b33-49fb-868d-a0600a047dec None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.244835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 36/76] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:07 2022] GET /identity/v3/users/742bb1241cf54598b570a2fd87179834 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:07.245565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4b24b-9b33-49fb-868d-a0600a047dec None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.275366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:07.281505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:07.318571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0b068-535a-41f4-89fe-d2c917cc5db4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.345237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4b24b-9b33-49fb-868d-a0600a047dec None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.352681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931583b-d4e9-4e48-bebb-10f310760ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.353306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931583b-d4e9-4e48-bebb-10f310760ead None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.353590 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931583b-d4e9-4e48-bebb-10f310760ead None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.353900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931583b-d4e9-4e48-bebb-10f310760ead None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.355618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931583b-d4e9-4e48-bebb-10f310760ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.357119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931583b-d4e9-4e48-bebb-10f310760ead None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.359136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 41/77] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:07.359729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931583b-d4e9-4e48-bebb-10f310760ead None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.379575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a077d88-9e00-4015-84cd-61dfb7548968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.380221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a077d88-9e00-4015-84cd-61dfb7548968 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.380528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a077d88-9e00-4015-84cd-61dfb7548968 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.380804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a077d88-9e00-4015-84cd-61dfb7548968 None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c/users/94f198dd02e44352b78185bbf6b74125/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.382740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a077d88-9e00-4015-84cd-61dfb7548968 None admin] RBAC: Authorizing `identity:create_grant(project_id=add9f115b9a84620af8483f8863c3f6c, user_id=94f198dd02e44352b78185bbf6b74125, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.384179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a077d88-9e00-4015-84cd-61dfb7548968 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.456372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931583b-d4e9-4e48-bebb-10f310760ead None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.457033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 37/78] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:07 2022] PUT /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c/users/94f198dd02e44352b78185bbf6b74125/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:07.457894 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a077d88-9e00-4015-84cd-61dfb7548968 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.493727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:07.506136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6be6f-94b2-4dd8-98db-3424baea82f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.506792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6be6f-94b2-4dd8-98db-3424baea82f6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.507330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6be6f-94b2-4dd8-98db-3424baea82f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.507625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6be6f-94b2-4dd8-98db-3424baea82f6 None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.508885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6be6f-94b2-4dd8-98db-3424baea82f6 None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.510993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6be6f-94b2-4dd8-98db-3424baea82f6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.512795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 42/79] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:07 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:07.513766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6be6f-94b2-4dd8-98db-3424baea82f6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.574457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:07.583471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a077d88-9e00-4015-84cd-61dfb7548968 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.624898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdf390-4eed-488e-b612-c589fa34541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.625563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf390-4eed-488e-b612-c589fa34541b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.625956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf390-4eed-488e-b612-c589fa34541b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.626595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdf390-4eed-488e-b612-c589fa34541b None admin] PATH_INFO: `/v3/users/94f198dd02e44352b78185bbf6b74125` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.627688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf390-4eed-488e-b612-c589fa34541b None admin] RBAC: Authorizing `identity:get_user(user_id=94f198dd02e44352b78185bbf6b74125)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.629462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdf390-4eed-488e-b612-c589fa34541b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.632062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 38/80] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:07 2022] GET /identity/v3/users/94f198dd02e44352b78185bbf6b74125 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:07.633014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdf390-4eed-488e-b612-c589fa34541b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.634747 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6be6f-94b2-4dd8-98db-3424baea82f6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.675280 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0c2aa-1b88-48ae-9602-3c463a93892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.675907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c2aa-1b88-48ae-9602-3c463a93892b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.676194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c2aa-1b88-48ae-9602-3c463a93892b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.676503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c2aa-1b88-48ae-9602-3c463a93892b None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.677660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0c2aa-1b88-48ae-9602-3c463a93892b None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.679101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0c2aa-1b88-48ae-9602-3c463a93892b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.681066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 43/81] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:07 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:07.681933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0c2aa-1b88-48ae-9602-3c463a93892b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.743890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:07.747619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdf390-4eed-488e-b612-c589fa34541b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.780956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7c990-7d31-4553-930b-8f4fe9fa399b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.781583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7c990-7d31-4553-930b-8f4fe9fa399b None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.781843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7c990-7d31-4553-930b-8f4fe9fa399b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.782109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7c990-7d31-4553-930b-8f4fe9fa399b None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.782757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7c990-7d31-4553-930b-8f4fe9fa399b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.784195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7c990-7d31-4553-930b-8f4fe9fa399b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.784392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0c2aa-1b88-48ae-9602-3c463a93892b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:07.819431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b27d9-5c8b-4b11-b27c-58d6ded3e1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:07.820086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b27d9-5c8b-4b11-b27c-58d6ded3e1ac None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:07.820405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b27d9-5c8b-4b11-b27c-58d6ded3e1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:07.820683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b27d9-5c8b-4b11-b27c-58d6ded3e1ac None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c/users/94f198dd02e44352b78185bbf6b74125/roles/23a3d7681bf247468a0baa9bfed6f4e9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:07.822862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b27d9-5c8b-4b11-b27c-58d6ded3e1ac None admin] RBAC: Authorizing `identity:create_grant(project_id=add9f115b9a84620af8483f8863c3f6c, user_id=94f198dd02e44352b78185bbf6b74125, role_id=23a3d7681bf247468a0baa9bfed6f4e9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:07.824319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b27d9-5c8b-4b11-b27c-58d6ded3e1ac None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:07.858080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 39/82] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:07 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:07.858080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7c990-7d31-4553-930b-8f4fe9fa399b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:07.888494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:07.913405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 44/83] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:07 2022] PUT /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c/users/94f198dd02e44352b78185bbf6b74125/roles/23a3d7681bf247468a0baa9bfed6f4e9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:07.914409 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b27d9-5c8b-4b11-b27c-58d6ded3e1ac None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:08.049090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7c990-7d31-4553-930b-8f4fe9fa399b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:08.121030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b27d9-5c8b-4b11-b27c-58d6ded3e1ac None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:08.123003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61ba75-3e54-4dc8-b643-80a175f48d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:08.124408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61ba75-3e54-4dc8-b643-80a175f48d6d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:08.124408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61ba75-3e54-4dc8-b643-80a175f48d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:08.124408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61ba75-3e54-4dc8-b643-80a175f48d6d None admin] PATH_INFO: `/v3/projects/ea0e3992febe4cc6a56494906093d44c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:08.125372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61ba75-3e54-4dc8-b643-80a175f48d6d None admin] RBAC: Authorizing `identity:get_project(project_id=ea0e3992febe4cc6a56494906093d44c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:08.126826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61ba75-3e54-4dc8-b643-80a175f48d6d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:08.128540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 40/84] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:07 2022] GET /identity/v3/projects/ea0e3992febe4cc6a56494906093d44c => generated 312 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:08.134709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61ba75-3e54-4dc8-b643-80a175f48d6d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:08.142709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:08.158638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1f0a1-a0cd-4cc7-b0a3-9efd8560bce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:08.159314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1f0a1-a0cd-4cc7-b0a3-9efd8560bce4 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:08.159636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1f0a1-a0cd-4cc7-b0a3-9efd8560bce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:08.159914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1f0a1-a0cd-4cc7-b0a3-9efd8560bce4 None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c/users/742bb1241cf54598b570a2fd87179834/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:08.161892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1f0a1-a0cd-4cc7-b0a3-9efd8560bce4 None admin] RBAC: Authorizing `identity:create_grant(project_id=add9f115b9a84620af8483f8863c3f6c, user_id=742bb1241cf54598b570a2fd87179834, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:08.163648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1f0a1-a0cd-4cc7-b0a3-9efd8560bce4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:08.235838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 45/85] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:07 2022] PUT /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c/users/742bb1241cf54598b570a2fd87179834/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:08.238214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:08.255027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61ba75-3e54-4dc8-b643-80a175f48d6d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:08.298777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d410a-c7d3-48f4-bbb9-3cf1838de13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:08.299676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d410a-c7d3-48f4-bbb9-3cf1838de13c None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:08.299906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d410a-c7d3-48f4-bbb9-3cf1838de13c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:08.300188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d410a-c7d3-48f4-bbb9-3cf1838de13c None admin] PATH_INFO: `/v3/projects/ea0e3992febe4cc6a56494906093d44c/users/94f198dd02e44352b78185bbf6b74125/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:08.302241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d410a-c7d3-48f4-bbb9-3cf1838de13c None admin] RBAC: Authorizing `identity:create_grant(project_id=ea0e3992febe4cc6a56494906093d44c, user_id=94f198dd02e44352b78185bbf6b74125, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:08.303779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d410a-c7d3-48f4-bbb9-3cf1838de13c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:08.395241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 41/86] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:08 2022] PUT /identity/v3/projects/ea0e3992febe4cc6a56494906093d44c/users/94f198dd02e44352b78185bbf6b74125/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:07:09.943629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875ab51-c740-4cec-ac30-2add5c21038b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:09.944170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875ab51-c740-4cec-ac30-2add5c21038b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:09.944170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875ab51-c740-4cec-ac30-2add5c21038b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:09.945581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 46/87] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:09.951041 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8868b5b-722e-45da-ad13-13a29de4a935 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:09.951441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8868b5b-722e-45da-ad13-13a29de4a935 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:09.951637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8868b5b-722e-45da-ad13-13a29de4a935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:09.971701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e8868b5b-722e-45da-ad13-13a29de4a935 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:09.972909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e8868b5b-722e-45da-ad13-13a29de4a935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:10.055957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 42/88] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:10.591603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875ab51-c740-4cec-ac30-2add5c21038b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:10.654219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca706890-01f6-400d-b4f0-ef23f3568467 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:10.654532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca706890-01f6-400d-b4f0-ef23f3568467 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:10.654792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca706890-01f6-400d-b4f0-ef23f3568467 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:10.656452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 43/89] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:10.668824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68772eb-0331-496c-8f96-412737a5adde None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:10.669114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68772eb-0331-496c-8f96-412737a5adde None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:10.669397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68772eb-0331-496c-8f96-412737a5adde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:10.701868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a68772eb-0331-496c-8f96-412737a5adde None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:10.703119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a68772eb-0331-496c-8f96-412737a5adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:10.787082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875ab51-c740-4cec-ac30-2add5c21038b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:10.803828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 44/90] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:10.858030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affc430-e464-41fa-8c24-d2101ec772ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:10.858625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affc430-e464-41fa-8c24-d2101ec772ef None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:10.858911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affc430-e464-41fa-8c24-d2101ec772ef None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:10.859186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affc430-e464-41fa-8c24-d2101ec772ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:10.874972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affc430-e464-41fa-8c24-d2101ec772ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9affc430-e464-41fa-8c24-d2101ec772ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:10.880216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:10.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:10.881970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:10.885995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 47/91] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:10.888158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68772eb-0331-496c-8f96-412737a5adde None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:10.904115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:11.042232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68772eb-0331-496c-8f96-412737a5adde None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:11.088632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:11.089267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.089552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.089827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.090503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:11.090805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:11.092202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:11.101403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 45/92] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:11.109796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affc430-e464-41fa-8c24-d2101ec772ef None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:11.186303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:11.263155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affc430-e464-41fa-8c24-d2101ec772ef None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:11.283907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:11.311712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009df57-14a2-480c-b161-e2f55a545ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:11.312718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009df57-14a2-480c-b161-e2f55a545ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.313237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009df57-14a2-480c-b161-e2f55a545ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.313712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009df57-14a2-480c-b161-e2f55a545ba2 None admin] PATH_INFO: `/v3/users/94f198dd02e44352b78185bbf6b74125` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.315028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009df57-14a2-480c-b161-e2f55a545ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=94f198dd02e44352b78185bbf6b74125)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:11.316705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009df57-14a2-480c-b161-e2f55a545ba2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:11.324369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 48/93] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:11 2022] GET /identity/v3/users/94f198dd02e44352b78185bbf6b74125 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:11.344315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009df57-14a2-480c-b161-e2f55a545ba2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:11.404663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:11.472479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe71bbd-287d-4069-b6db-f260f5599c0a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:11.528024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb410c-4251-4777-b0ab-14d9b8fe29ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:11.528696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb410c-4251-4777-b0ab-14d9b8fe29ce None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.528974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb410c-4251-4777-b0ab-14d9b8fe29ce None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.529252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb410c-4251-4777-b0ab-14d9b8fe29ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.562256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb410c-4251-4777-b0ab-14d9b8fe29ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c8fb410c-4251-4777-b0ab-14d9b8fe29ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:11.571672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:11.572694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:11.573924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 46/94] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:07:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:11.574751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551c5be-9a25-473f-bbf7-5b89bcc5b351 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.575118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551c5be-9a25-473f-bbf7-5b89bcc5b351 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.581161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551c5be-9a25-473f-bbf7-5b89bcc5b351 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.583468 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 47/95] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:11.590561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551c5be-9a25-473f-bbf7-5b89bcc5b351 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:11.597475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009df57-14a2-480c-b161-e2f55a545ba2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:11.617300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:11.665426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db525a-6c6a-488c-9deb-34d8db2efe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:11.666934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db525a-6c6a-488c-9deb-34d8db2efe15 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.668033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db525a-6c6a-488c-9deb-34d8db2efe15 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.669302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db525a-6c6a-488c-9deb-34d8db2efe15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.750732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551c5be-9a25-473f-bbf7-5b89bcc5b351 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:11.758726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db525a-6c6a-488c-9deb-34d8db2efe15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:11.760715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db525a-6c6a-488c-9deb-34d8db2efe15 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:11.778106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:11.785797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a94f70-0d71-4d48-8258-3ae96c4bc50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:11.786549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94f70-0d71-4d48-8258-3ae96c4bc50b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.786845 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94f70-0d71-4d48-8258-3ae96c4bc50b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.787117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94f70-0d71-4d48-8258-3ae96c4bc50b None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.787771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a94f70-0d71-4d48-8258-3ae96c4bc50b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:11.788073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a94f70-0d71-4d48-8258-3ae96c4bc50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:11.789472 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a94f70-0d71-4d48-8258-3ae96c4bc50b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:11.798066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 48/96] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:11.802814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3c6f3-33a3-4278-a6ca-ea9fe2e835be None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.802814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3c6f3-33a3-4278-a6ca-ea9fe2e835be None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.802814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3c6f3-33a3-4278-a6ca-ea9fe2e835be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.817299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-27e3c6f3-33a3-4278-a6ca-ea9fe2e835be None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:11.818485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27e3c6f3-33a3-4278-a6ca-ea9fe2e835be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:11.862187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 49/97] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 13:07:11.863432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4bef4-3f71-46d4-bb20-776e654e90c3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.863720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4bef4-3f71-46d4-bb20-776e654e90c3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.863990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4bef4-3f71-46d4-bb20-776e654e90c3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.865284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 50/98] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:11.869452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b768-42e7-445d-b4fc-ae154db18c9b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.869738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b768-42e7-445d-b4fc-ae154db18c9b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.870024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b768-42e7-445d-b4fc-ae154db18c9b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.871276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 51/99] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:11.873532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b4e6-6441-4fbf-8146-4926e970c763 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:11.873821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b4e6-6441-4fbf-8146-4926e970c763 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:11.874422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b4e6-6441-4fbf-8146-4926e970c763 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:11.875638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 52/100] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:11.876358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5b4e6-6441-4fbf-8146-4926e970c763 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:11.884780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 49/101] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:11.885648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3c6f3-33a3-4278-a6ca-ea9fe2e835be None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:11.898706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:11.988636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3c6f3-33a3-4278-a6ca-ea9fe2e835be None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:11.991794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5b4e6-6441-4fbf-8146-4926e970c763 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:12.010053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:12.024533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbbb5c-cb0b-4db9-a65e-61faeb65274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.025205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbbb5c-cb0b-4db9-a65e-61faeb65274a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.025993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbbb5c-cb0b-4db9-a65e-61faeb65274a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.026268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbbb5c-cb0b-4db9-a65e-61faeb65274a None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.027426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbbb5c-cb0b-4db9-a65e-61faeb65274a None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:12.027659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6e04c-3d39-49d3-82e8-979b165f58b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.028463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6e04c-3d39-49d3-82e8-979b165f58b8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.028886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6e04c-3d39-49d3-82e8-979b165f58b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.029016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbbb5c-cb0b-4db9-a65e-61faeb65274a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:12.029415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6e04c-3d39-49d3-82e8-979b165f58b8 None admin] PATH_INFO: `/v3/users/94f198dd02e44352b78185bbf6b74125` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.030658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6e04c-3d39-49d3-82e8-979b165f58b8 None admin] RBAC: Authorizing `identity:get_user(user_id=94f198dd02e44352b78185bbf6b74125)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:12.032221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6e04c-3d39-49d3-82e8-979b165f58b8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:12.036312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 53/102] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:11 2022] GET /identity/v3/users/94f198dd02e44352b78185bbf6b74125 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:12.036423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 50/103] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:11 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:12.036705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0c3a4-d3dd-4e0d-9514-d85bdb7c235a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.036998 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0c3a4-d3dd-4e0d-9514-d85bdb7c235a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.037279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0c3a4-d3dd-4e0d-9514-d85bdb7c235a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.039350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca46663-7ff6-453a-b2c4-a7bdae65aec0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.039635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca46663-7ff6-453a-b2c4-a7bdae65aec0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.039910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca46663-7ff6-453a-b2c4-a7bdae65aec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.055721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-60f0c3a4-d3dd-4e0d-9514-d85bdb7c235a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:12.057010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-60f0c3a4-d3dd-4e0d-9514-d85bdb7c235a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:12.057993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4ca46663-7ff6-453a-b2c4-a7bdae65aec0 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:12.059288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4ca46663-7ff6-453a-b2c4-a7bdae65aec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:12.130968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 54/104] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:12.144530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 51/105] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:12.144530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0c3a4-d3dd-4e0d-9514-d85bdb7c235a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.144730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0ce2-d66e-47ff-a55a-56b6f9783a91 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.144730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0ce2-d66e-47ff-a55a-56b6f9783a91 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.144730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab0ce2-d66e-47ff-a55a-56b6f9783a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.155325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:12.161505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-cdab0ce2-d66e-47ff-a55a-56b6f9783a91 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:12.163224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cdab0ce2-d66e-47ff-a55a-56b6f9783a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:12.251215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 55/106] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:12.252228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab0ce2-d66e-47ff-a55a-56b6f9783a91 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.259433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0c3a4-d3dd-4e0d-9514-d85bdb7c235a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:12.281128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:12.299513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.300296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.300709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.301076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.301907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] RBAC: Adding query filter params (role.id=eb33c965778e405b95ebf794f31e329a, scope.project.id=add9f115b9a84620af8483f8863c3f6c, user.id=94f198dd02e44352b78185bbf6b74125) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:12.302367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:12.303996 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:12.312878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 52/107] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/role_assignments?user.id=94f198dd02e44352b78185bbf6b74125&scope.project.id=add9f115b9a84620af8483f8863c3f6c&role.id=eb33c965778e405b95ebf794f31e329a => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:12.313980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.439422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab0ce2-d66e-47ff-a55a-56b6f9783a91 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:12.477922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc25c8-aa98-4c43-b423-2ba0ddd31abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.478938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc25c8-aa98-4c43-b423-2ba0ddd31abd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.479469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc25c8-aa98-4c43-b423-2ba0ddd31abd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.479942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc25c8-aa98-4c43-b423-2ba0ddd31abd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.484658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:12.494444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423c8f6-6170-4328-82b7-c785cece85ca None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:12.559038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ffe4c-f878-486c-be21-368bad093985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.559587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ffe4c-f878-486c-be21-368bad093985 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.559862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ffe4c-f878-486c-be21-368bad093985 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.560133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ffe4c-f878-486c-be21-368bad093985 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.573439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ffe4c-f878-486c-be21-368bad093985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:12.574896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc25c8-aa98-4c43-b423-2ba0ddd31abd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:12.576457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc25c8-aa98-4c43-b423-2ba0ddd31abd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2b1ffe4c-f878-486c-be21-368bad093985 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:12.578671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:12.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:12.580439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 53/108] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:12.581086 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ffe4c-f878-486c-be21-368bad093985 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.637655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:12.688290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 56/109] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:12.693142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc25c8-aa98-4c43-b423-2ba0ddd31abd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.693997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ffe4c-f878-486c-be21-368bad093985 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:12.727341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:12.729395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc862e-163a-4e0e-8edf-4d23e04ebe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.730016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc862e-163a-4e0e-8edf-4d23e04ebe8e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.730291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc862e-163a-4e0e-8edf-4d23e04ebe8e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.730556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc862e-163a-4e0e-8edf-4d23e04ebe8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.740827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc862e-163a-4e0e-8edf-4d23e04ebe8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-68bc862e-163a-4e0e-8edf-4d23e04ebe8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:12.746073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:12.747063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 54/110] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:12.747907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc862e-163a-4e0e-8edf-4d23e04ebe8e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.794744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc25c8-aa98-4c43-b423-2ba0ddd31abd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:12.827517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd46e5a-9742-43ea-9040-83470da62417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.828075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46e5a-9742-43ea-9040-83470da62417 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.828392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46e5a-9742-43ea-9040-83470da62417 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.828670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd46e5a-9742-43ea-9040-83470da62417 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.838878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd46e5a-9742-43ea-9040-83470da62417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1cd46e5a-9742-43ea-9040-83470da62417 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:12.843607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:12.844744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:12.845527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 57/111] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:12.846184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd46e5a-9742-43ea-9040-83470da62417 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.851789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc862e-163a-4e0e-8edf-4d23e04ebe8e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:12.853366 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:12.885422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.886022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.886301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.886568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.887278 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:12.887582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:12.888977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:12.901050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 55/112] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:12.901862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.938157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:12.943876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd46e5a-9742-43ea-9040-83470da62417 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:12.978283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8367a-284a-4371-8bcc-ed816e17d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:12.978924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8367a-284a-4371-8bcc-ed816e17d2ab None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:12.979201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8367a-284a-4371-8bcc-ed816e17d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:12.979471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8367a-284a-4371-8bcc-ed816e17d2ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:12.980579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8367a-284a-4371-8bcc-ed816e17d2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:12.982029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8367a-284a-4371-8bcc-ed816e17d2ab None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:12.983571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 58/113] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:12.984343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8367a-284a-4371-8bcc-ed816e17d2ab None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:12.999531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72de3a4-82a6-401d-93e6-85583ec0281d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.033059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab43104-7191-494e-9844-57400c90440e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.033674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43104-7191-494e-9844-57400c90440e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.033954 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43104-7191-494e-9844-57400c90440e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.034226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43104-7191-494e-9844-57400c90440e None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.035328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab43104-7191-494e-9844-57400c90440e None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.036783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab43104-7191-494e-9844-57400c90440e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.038378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 56/114] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:13.038378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab43104-7191-494e-9844-57400c90440e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.074025 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.081792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8367a-284a-4371-8bcc-ed816e17d2ab None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.116240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.116872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.117149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.117420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.118074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:13.118375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.119760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.127840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 59/115] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:13.128681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.143303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab43104-7191-494e-9844-57400c90440e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.155600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.176586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.177198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.177475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.177750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.178397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:13.178697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.180068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.188117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 57/116] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:13.188904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.226958 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22a85d-12b5-41d7-bcca-18814e814aa6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.260423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba9e92-eebc-4583-9c96-518b58be130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.261021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9e92-eebc-4583-9c96-518b58be130a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.261300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9e92-eebc-4583-9c96-518b58be130a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.261574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba9e92-eebc-4583-9c96-518b58be130a None admin] PATH_INFO: `/v3/users/742bb1241cf54598b570a2fd87179834` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.262685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba9e92-eebc-4583-9c96-518b58be130a None admin] RBAC: Authorizing `identity:get_user(user_id=742bb1241cf54598b570a2fd87179834)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.264128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba9e92-eebc-4583-9c96-518b58be130a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.265720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 60/117] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/users/742bb1241cf54598b570a2fd87179834 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:13.266476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba9e92-eebc-4583-9c96-518b58be130a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.285588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.292593 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34510780-e735-4b46-b9c4-af3e44ff6852 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.327912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67426a95-5ae0-4deb-9f21-b645ef456107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.328536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67426a95-5ae0-4deb-9f21-b645ef456107 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.328821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67426a95-5ae0-4deb-9f21-b645ef456107 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.329095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67426a95-5ae0-4deb-9f21-b645ef456107 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.329739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67426a95-5ae0-4deb-9f21-b645ef456107 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.331180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67426a95-5ae0-4deb-9f21-b645ef456107 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.360159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.361968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba9e92-eebc-4583-9c96-518b58be130a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.394751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.395337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.395616 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.395884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.396622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] RBAC: Adding query filter params (role.id=23a3d7681bf247468a0baa9bfed6f4e9, scope.project.id=add9f115b9a84620af8483f8863c3f6c, user.id=94f198dd02e44352b78185bbf6b74125) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:13.396928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.398573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.404172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 61/118] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/role_assignments?user.id=94f198dd02e44352b78185bbf6b74125&scope.project.id=add9f115b9a84620af8483f8863c3f6c&role.id=23a3d7681bf247468a0baa9bfed6f4e9 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:13.405015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.406776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 58/119] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:07:13 2022] POST /identity/v3/projects => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:13.407810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67426a95-5ae0-4deb-9f21-b645ef456107 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.534092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67426a95-5ae0-4deb-9f21-b645ef456107 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.539572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3e329-91a5-4bbb-add6-623fd9a2976c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.579080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6e6bf-b92a-4cd3-a27f-1eeec6241d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.579987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e6bf-b92a-4cd3-a27f-1eeec6241d99 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.580745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e6bf-b92a-4cd3-a27f-1eeec6241d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.581026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6e6bf-b92a-4cd3-a27f-1eeec6241d99 None admin] PATH_INFO: `/v3/users/94f198dd02e44352b78185bbf6b74125` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.582697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6e6bf-b92a-4cd3-a27f-1eeec6241d99 None admin] RBAC: Authorizing `identity:get_user(user_id=94f198dd02e44352b78185bbf6b74125)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.584250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6e6bf-b92a-4cd3-a27f-1eeec6241d99 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.585529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8325b6-3a50-45c8-99b2-0acde90351be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.586034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 59/120] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/users/94f198dd02e44352b78185bbf6b74125 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:13.586392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8325b6-3a50-45c8-99b2-0acde90351be None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.586751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8325b6-3a50-45c8-99b2-0acde90351be None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.586866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6e6bf-b92a-4cd3-a27f-1eeec6241d99 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.587259 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8325b6-3a50-45c8-99b2-0acde90351be None admin] PATH_INFO: `/v3/users/0b89a27630df4ca3a2f3fd38d83b689f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.610280 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8325b6-3a50-45c8-99b2-0acde90351be None admin] RBAC: Authorizing `identity:get_user(user_id=0b89a27630df4ca3a2f3fd38d83b689f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.611556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.611672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8325b6-3a50-45c8-99b2-0acde90351be None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.613786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 62/121] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/users/0b89a27630df4ca3a2f3fd38d83b689f => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:13.614755 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8325b6-3a50-45c8-99b2-0acde90351be None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.636707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.728904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6e6bf-b92a-4cd3-a27f-1eeec6241d99 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.739607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8325b6-3a50-45c8-99b2-0acde90351be None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:13.764037 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e80de-a5e1-444b-810e-b68f1d434796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.764705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e80de-a5e1-444b-810e-b68f1d434796 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.764970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e80de-a5e1-444b-810e-b68f1d434796 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.765251 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e80de-a5e1-444b-810e-b68f1d434796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.776206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de31fa7-249c-49ee-89fa-75641d5cb80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:13.777137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de31fa7-249c-49ee-89fa-75641d5cb80b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.777588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de31fa7-249c-49ee-89fa-75641d5cb80b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.778112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de31fa7-249c-49ee-89fa-75641d5cb80b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.835037 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e80de-a5e1-444b-810e-b68f1d434796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.836551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e80de-a5e1-444b-810e-b68f1d434796 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.848401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de31fa7-249c-49ee-89fa-75641d5cb80b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:13.850158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de31fa7-249c-49ee-89fa-75641d5cb80b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:13.851379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.870457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.944517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 60/122] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:13.945211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e80de-a5e1-444b-810e-b68f1d434796 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:13.959155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 63/123] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:13.959994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de31fa7-249c-49ee-89fa-75641d5cb80b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:14.057493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e80de-a5e1-444b-810e-b68f1d434796 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:14.063480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de31fa7-249c-49ee-89fa-75641d5cb80b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:14.071853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:14.079698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:14.097856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ed480-2435-438b-95db-e1d7372e52c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:14.098488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ed480-2435-438b-95db-e1d7372e52c9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:14.098771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ed480-2435-438b-95db-e1d7372e52c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:14.099045 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ed480-2435-438b-95db-e1d7372e52c9 None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:14.099579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48227a83-bc5b-48b9-962f-26eeba80470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:14.100158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ed480-2435-438b-95db-e1d7372e52c9 None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:14.100260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48227a83-bc5b-48b9-962f-26eeba80470a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:14.100515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48227a83-bc5b-48b9-962f-26eeba80470a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:14.100796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48227a83-bc5b-48b9-962f-26eeba80470a None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:14.101676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ed480-2435-438b-95db-e1d7372e52c9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:14.102298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48227a83-bc5b-48b9-962f-26eeba80470a None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:14.103657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 64/124] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:14.104766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ed480-2435-438b-95db-e1d7372e52c9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:14.109342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48227a83-bc5b-48b9-962f-26eeba80470a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:14.114128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 61/125] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:13 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:14.114901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48227a83-bc5b-48b9-962f-26eeba80470a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:14.217453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ed480-2435-438b-95db-e1d7372e52c9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:14.221265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48227a83-bc5b-48b9-962f-26eeba80470a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:14.252700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca9299-6e16-4d7b-be47-0277dc03b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:14.253339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca9299-6e16-4d7b-be47-0277dc03b73e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:14.253626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca9299-6e16-4d7b-be47-0277dc03b73e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:14.253948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca9299-6e16-4d7b-be47-0277dc03b73e None admin] PATH_INFO: `/v3/projects/ea0e3992febe4cc6a56494906093d44c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:14.255127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca9299-6e16-4d7b-be47-0277dc03b73e None admin] RBAC: Authorizing `identity:get_project(project_id=ea0e3992febe4cc6a56494906093d44c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:14.256457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:14.256713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca9299-6e16-4d7b-be47-0277dc03b73e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:14.257058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:14.257337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:14.257607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:14.258544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=add9f115b9a84620af8483f8863c3f6c, user.id=742bb1241cf54598b570a2fd87179834) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:14.258654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 65/126] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:14 2022] GET /identity/v3/projects/ea0e3992febe4cc6a56494906093d44c => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:14.258727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:14.260359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:14.261286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca9299-6e16-4d7b-be47-0277dc03b73e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:14.267321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 62/127] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:14 2022] GET /identity/v3/role_assignments?user.id=742bb1241cf54598b570a2fd87179834&scope.project.id=add9f115b9a84620af8483f8863c3f6c&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:14.268122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:14.321065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:14.324103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:14.378661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca9299-6e16-4d7b-be47-0277dc03b73e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:14.402630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b98fe-cf83-4064-af26-f23203027f78 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:14.424891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a7f51-56d8-4174-bf78-03246027ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:14.425567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a7f51-56d8-4174-bf78-03246027ecd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:14.425973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a7f51-56d8-4174-bf78-03246027ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:14.426252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a7f51-56d8-4174-bf78-03246027ecd6 None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c/users/0b89a27630df4ca3a2f3fd38d83b689f/roles/ab3629eff4074dd29639a73114e696d4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:14.428245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a7f51-56d8-4174-bf78-03246027ecd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=add9f115b9a84620af8483f8863c3f6c, user_id=0b89a27630df4ca3a2f3fd38d83b689f, role_id=ab3629eff4074dd29639a73114e696d4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:14.429756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a7f51-56d8-4174-bf78-03246027ecd6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:14.441635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba294789-2fdc-4867-96b5-190917b39ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:14.442256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294789-2fdc-4867-96b5-190917b39ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:14.442546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294789-2fdc-4867-96b5-190917b39ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:14.442824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba294789-2fdc-4867-96b5-190917b39ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:14.446343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba294789-2fdc-4867-96b5-190917b39ff8 None admin] RBAC: Adding query filter params (role.id=eb33c965778e405b95ebf794f31e329a, scope.project.id=ea0e3992febe4cc6a56494906093d44c, user.id=94f198dd02e44352b78185bbf6b74125) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:14.446343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba294789-2fdc-4867-96b5-190917b39ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:14.446343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba294789-2fdc-4867-96b5-190917b39ff8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:14.452611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 63/128] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:14 2022] GET /identity/v3/role_assignments?user.id=94f198dd02e44352b78185bbf6b74125&scope.project.id=ea0e3992febe4cc6a56494906093d44c&role.id=eb33c965778e405b95ebf794f31e329a => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:14.510782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 66/129] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:14 2022] PUT /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c/users/0b89a27630df4ca3a2f3fd38d83b689f/roles/ab3629eff4074dd29639a73114e696d4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:15.677114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cfef0-37aa-4f45-8519-8e164d1ab256 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:15.677661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cfef0-37aa-4f45-8519-8e164d1ab256 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:15.677661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cfef0-37aa-4f45-8519-8e164d1ab256 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:15.679014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 64/130] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:15.684207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a5191-3600-4502-ace5-f238020e6095 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:15.684677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a5191-3600-4502-ace5-f238020e6095 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:15.684799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a5191-3600-4502-ace5-f238020e6095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:15.702655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4e4a5191-3600-4502-ace5-f238020e6095 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:15.703880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4e4a5191-3600-4502-ace5-f238020e6095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:15.772033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 67/131] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:16.041641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cfef0-37aa-4f45-8519-8e164d1ab256 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:16.144988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cfef0-37aa-4f45-8519-8e164d1ab256 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:16.156035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:16.177980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386250b5-3cf3-41e9-97e5-0b17c05184e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:16.178609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386250b5-3cf3-41e9-97e5-0b17c05184e4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:16.178892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386250b5-3cf3-41e9-97e5-0b17c05184e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:16.179172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386250b5-3cf3-41e9-97e5-0b17c05184e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:16.180263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386250b5-3cf3-41e9-97e5-0b17c05184e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:16.181782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386250b5-3cf3-41e9-97e5-0b17c05184e4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:16.183344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 65/132] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:16.188958 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a5191-3600-4502-ace5-f238020e6095 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:16.230953 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:16.284626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a5191-3600-4502-ace5-f238020e6095 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:16.317952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273594a-a743-47b9-9801-7ee9015b2a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:16.318482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273594a-a743-47b9-9801-7ee9015b2a61 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:16.318761 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273594a-a743-47b9-9801-7ee9015b2a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:16.319031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273594a-a743-47b9-9801-7ee9015b2a61 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:16.319683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273594a-a743-47b9-9801-7ee9015b2a61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:16.321145 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273594a-a743-47b9-9801-7ee9015b2a61 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:16.411033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 68/133] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:16 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:16.653732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1942c5-a210-482c-bf2d-279d34c26674 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:16.653918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1942c5-a210-482c-bf2d-279d34c26674 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:16.654155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1942c5-a210-482c-bf2d-279d34c26674 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:16.655507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 66/134] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:16.661118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b1a94-5e2d-4467-a3cc-a6281f372414 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:16.661759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b1a94-5e2d-4467-a3cc-a6281f372414 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:16.662052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b1a94-5e2d-4467-a3cc-a6281f372414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:16.681451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3b8b1a94-5e2d-4467-a3cc-a6281f372414 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:16.683195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3b8b1a94-5e2d-4467-a3cc-a6281f372414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:16.752694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 69/135] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:17.026417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1942c5-a210-482c-bf2d-279d34c26674 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:17.132869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1942c5-a210-482c-bf2d-279d34c26674 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:17.167956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d99a9-1f63-4c03-a4d5-822139049c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:17.168618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d99a9-1f63-4c03-a4d5-822139049c84 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:17.168953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d99a9-1f63-4c03-a4d5-822139049c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:17.169232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d99a9-1f63-4c03-a4d5-822139049c84 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:17.179715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d99a9-1f63-4c03-a4d5-822139049c84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-023d99a9-1f63-4c03-a4d5-822139049c84 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:17.185191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:17.186072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:17.186880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 67/136] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:17.192169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b1a94-5e2d-4467-a3cc-a6281f372414 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:17.220889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:17.295889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b1a94-5e2d-4467-a3cc-a6281f372414 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:17.330886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:17.331488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:17.331767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:17.332038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:17.332722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:17.333021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:17.334457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:17.342644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 70/137] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:17.348640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d99a9-1f63-4c03-a4d5-822139049c84 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:17.393815 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:17.456401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d99a9-1f63-4c03-a4d5-822139049c84 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:17.491981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196c55-b620-49ea-b7ec-13a7501a791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:17.492695 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196c55-b620-49ea-b7ec-13a7501a791c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:17.493046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196c55-b620-49ea-b7ec-13a7501a791c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:17.493324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196c55-b620-49ea-b7ec-13a7501a791c None admin] PATH_INFO: `/v3/users/0b89a27630df4ca3a2f3fd38d83b689f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:17.495054 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72196c55-b620-49ea-b7ec-13a7501a791c None admin] RBAC: Authorizing `identity:get_user(user_id=0b89a27630df4ca3a2f3fd38d83b689f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:17.496576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72196c55-b620-49ea-b7ec-13a7501a791c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:17.498147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 68/138] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:17 2022] GET /identity/v3/users/0b89a27630df4ca3a2f3fd38d83b689f => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:17.503852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:17.597785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:17.608635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449c51b-4659-4eb2-adee-9fc483ca0f47 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:17.645559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1ded1-5bf6-4380-bd32-30d2a6ceac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:17.646166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1ded1-5bf6-4380-bd32-30d2a6ceac1c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:17.646447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1ded1-5bf6-4380-bd32-30d2a6ceac1c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:17.646720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1ded1-5bf6-4380-bd32-30d2a6ceac1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:17.714740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1ded1-5bf6-4380-bd32-30d2a6ceac1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:17.716208 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1ded1-5bf6-4380-bd32-30d2a6ceac1c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:17.812296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:17.822273 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 71/139] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:17.828048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196c55-b620-49ea-b7ec-13a7501a791c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:17.939142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196c55-b620-49ea-b7ec-13a7501a791c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:17.946421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:17.977787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d6161-9833-4787-ac3f-80a2f72fa26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:17.978439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d6161-9833-4787-ac3f-80a2f72fa26b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:17.979258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d6161-9833-4787-ac3f-80a2f72fa26b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:17.979538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d6161-9833-4787-ac3f-80a2f72fa26b None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:17.980708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d6161-9833-4787-ac3f-80a2f72fa26b None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:17.982146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d6161-9833-4787-ac3f-80a2f72fa26b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:17.984087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 69/140] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:17 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:17.990772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1ded1-5bf6-4380-bd32-30d2a6ceac1c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:18.095979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1ded1-5bf6-4380-bd32-30d2a6ceac1c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:18.140220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea17251-bf8a-44ce-857a-227739dd0292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:18.141233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea17251-bf8a-44ce-857a-227739dd0292 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:18.141728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea17251-bf8a-44ce-857a-227739dd0292 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:18.142197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea17251-bf8a-44ce-857a-227739dd0292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:18.143130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea17251-bf8a-44ce-857a-227739dd0292 None admin] RBAC: Adding query filter params (role.id=ab3629eff4074dd29639a73114e696d4, scope.project.id=add9f115b9a84620af8483f8863c3f6c, user.id=0b89a27630df4ca3a2f3fd38d83b689f) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:18.143634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea17251-bf8a-44ce-857a-227739dd0292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:18.145471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea17251-bf8a-44ce-857a-227739dd0292 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:18.151763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 72/141] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:17 2022] GET /identity/v3/role_assignments?user.id=0b89a27630df4ca3a2f3fd38d83b689f&scope.project.id=add9f115b9a84620af8483f8863c3f6c&role.id=ab3629eff4074dd29639a73114e696d4 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:18.669913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af271-af2c-44ff-a8e2-e3377d432fc4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:18.670429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af271-af2c-44ff-a8e2-e3377d432fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:18.670504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af271-af2c-44ff-a8e2-e3377d432fc4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:18.671813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 70/142] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:07:18.677310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7dad1-dc51-47e7-a763-98db529939f4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:18.677603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7dad1-dc51-47e7-a763-98db529939f4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:18.677879 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7dad1-dc51-47e7-a763-98db529939f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:18.696185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6dc7dad1-dc51-47e7-a763-98db529939f4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:18.697993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6dc7dad1-dc51-47e7-a763-98db529939f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:18.748471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:18.766194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 73/143] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:18.932229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aabbe-1171-4a11-be61-97a5d5227ebd None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:18.932538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aabbe-1171-4a11-be61-97a5d5227ebd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:18.932738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aabbe-1171-4a11-be61-97a5d5227ebd None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:18.934230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 71/144] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:18.939517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6ee8a-a18c-4c58-9369-154646ebe622 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:18.939951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6ee8a-a18c-4c58-9369-154646ebe622 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:18.940143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6ee8a-a18c-4c58-9369-154646ebe622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:18.948294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09275fb5-f31e-445b-8327-974506035731 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:18.948596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09275fb5-f31e-445b-8327-974506035731 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:18.948873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09275fb5-f31e-445b-8327-974506035731 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:18.950283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 72/145] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:18.955388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b905fac-08ea-4241-80e2-0c5f88032494 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:18.955839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b905fac-08ea-4241-80e2-0c5f88032494 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:18.956122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b905fac-08ea-4241-80e2-0c5f88032494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:18.959218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f8f6ee8a-a18c-4c58-9369-154646ebe622 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:18.960449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f8f6ee8a-a18c-4c58-9369-154646ebe622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:18.976459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4b905fac-08ea-4241-80e2-0c5f88032494 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:18.977625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4b905fac-08ea-4241-80e2-0c5f88032494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:19.037758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 74/146] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:19.039997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6ee8a-a18c-4c58-9369-154646ebe622 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.050340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 73/147] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:19.145534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6ee8a-a18c-4c58-9369-154646ebe622 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.160308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:19.179481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9528c5-3ac8-42a7-874f-f8481ff2db7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.182973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9528c5-3ac8-42a7-874f-f8481ff2db7e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.182973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9528c5-3ac8-42a7-874f-f8481ff2db7e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.182973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9528c5-3ac8-42a7-874f-f8481ff2db7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.190785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9528c5-3ac8-42a7-874f-f8481ff2db7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-eb9528c5-3ac8-42a7-874f-f8481ff2db7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:19.196128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:19.197964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:19.199244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:19.201841 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 75/148] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:19.207119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b905fac-08ea-4241-80e2-0c5f88032494 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.306021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9528c5-3ac8-42a7-874f-f8481ff2db7e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.306315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b905fac-08ea-4241-80e2-0c5f88032494 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.309958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:19.341801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.342402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.342677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.342945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.343847 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:19.344145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:19.345548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:19.353635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 74/149] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:19.354259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.406257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9528c5-3ac8-42a7-874f-f8481ff2db7e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.439629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da525e3c-a550-46e9-a70f-b5a52bb1185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.440180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da525e3c-a550-46e9-a70f-b5a52bb1185f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.440558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da525e3c-a550-46e9-a70f-b5a52bb1185f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.440768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da525e3c-a550-46e9-a70f-b5a52bb1185f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.441852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da525e3c-a550-46e9-a70f-b5a52bb1185f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:19.443300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da525e3c-a550-46e9-a70f-b5a52bb1185f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:19.444876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 76/150] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:19.445974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da525e3c-a550-46e9-a70f-b5a52bb1185f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.456990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdb78e-9856-41b7-a576-945d91bf1936 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.489230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:19.491105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5bf0e-bce0-4677-bd3d-0652a2b0c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.491721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5bf0e-bce0-4677-bd3d-0652a2b0c46d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.491997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5bf0e-bce0-4677-bd3d-0652a2b0c46d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.492263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5bf0e-bce0-4677-bd3d-0652a2b0c46d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.502705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5bf0e-bce0-4677-bd3d-0652a2b0c46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-93f5bf0e-bce0-4677-bd3d-0652a2b0c46d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:19.507962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:19.509004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 75/151] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:19.510398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5bf0e-bce0-4677-bd3d-0652a2b0c46d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.538811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:19.545495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da525e3c-a550-46e9-a70f-b5a52bb1185f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.580979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739f06b-6588-4739-8f93-d6247c99fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.581681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739f06b-6588-4739-8f93-d6247c99fc2c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.582126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739f06b-6588-4739-8f93-d6247c99fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.582478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739f06b-6588-4739-8f93-d6247c99fc2c None admin] PATH_INFO: `/v3/users/2bca075c6acb413f8998c6a57eca1ac0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.596007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739f06b-6588-4739-8f93-d6247c99fc2c None admin] RBAC: Authorizing `identity:get_user(user_id=2bca075c6acb413f8998c6a57eca1ac0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:19.597494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739f06b-6588-4739-8f93-d6247c99fc2c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:19.599091 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 77/152] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/users/2bca075c6acb413f8998c6a57eca1ac0 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:19.603900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739f06b-6588-4739-8f93-d6247c99fc2c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.612406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5bf0e-bce0-4677-bd3d-0652a2b0c46d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.646700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879f075-0228-4441-a17c-6d075898bc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.647297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f075-0228-4441-a17c-6d075898bc59 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.647569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f075-0228-4441-a17c-6d075898bc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.647835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f075-0228-4441-a17c-6d075898bc59 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.648498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879f075-0228-4441-a17c-6d075898bc59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:19.649918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879f075-0228-4441-a17c-6d075898bc59 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:19.695051 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:19.697809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739f06b-6588-4739-8f93-d6247c99fc2c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.713554 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 76/153] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:19 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:19.714374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879f075-0228-4441-a17c-6d075898bc59 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.735805 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.736466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.737043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.737503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.738241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:19.738556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:19.741346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:19.750866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 78/154] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:19.756949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.815155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:19.834613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879f075-0228-4441-a17c-6d075898bc59 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.874255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b60e0-e203-4287-a4f2-afdde6378495 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:19.877875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e67d5-8be3-482d-9874-dc395540df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.878562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e67d5-8be3-482d-9874-dc395540df1d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.878846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e67d5-8be3-482d-9874-dc395540df1d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.879122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e67d5-8be3-482d-9874-dc395540df1d None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.880361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e67d5-8be3-482d-9874-dc395540df1d None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:19.881837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5e67d5-8be3-482d-9874-dc395540df1d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:19.883492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 77/155] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:19.889125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e67d5-8be3-482d-9874-dc395540df1d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.916814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e0493-b84d-40ec-bc39-fc804e7fe825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:19.917610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0493-b84d-40ec-bc39-fc804e7fe825 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.918218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0493-b84d-40ec-bc39-fc804e7fe825 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.918499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0493-b84d-40ec-bc39-fc804e7fe825 None admin] PATH_INFO: `/v3/users/2bca075c6acb413f8998c6a57eca1ac0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.919778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e0493-b84d-40ec-bc39-fc804e7fe825 None admin] RBAC: Authorizing `identity:get_user(user_id=2bca075c6acb413f8998c6a57eca1ac0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:19.921272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e0493-b84d-40ec-bc39-fc804e7fe825 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:19.923173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 79/156] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/users/2bca075c6acb413f8998c6a57eca1ac0 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:19.929076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e0493-b84d-40ec-bc39-fc804e7fe825 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:19.965313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:20.007071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e67d5-8be3-482d-9874-dc395540df1d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:20.052867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c78f-3945-4988-af53-767b5e62d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:20.053524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c78f-3945-4988-af53-767b5e62d62e None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:20.053804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c78f-3945-4988-af53-767b5e62d62e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:20.054080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c78f-3945-4988-af53-767b5e62d62e None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d/users/2bca075c6acb413f8998c6a57eca1ac0/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:20.055959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c78f-3945-4988-af53-767b5e62d62e None admin] RBAC: Authorizing `identity:create_grant(project_id=312738b3aa6f46e2bab330e5c745742d, user_id=2bca075c6acb413f8998c6a57eca1ac0, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:20.056296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e0493-b84d-40ec-bc39-fc804e7fe825 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:20.057467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c78f-3945-4988-af53-767b5e62d62e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:20.098828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee515f8-0212-40e0-bf93-e5956fedcdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:20.099090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee515f8-0212-40e0-bf93-e5956fedcdcd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:20.099291 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee515f8-0212-40e0-bf93-e5956fedcdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:20.099562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee515f8-0212-40e0-bf93-e5956fedcdcd None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:20.101326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee515f8-0212-40e0-bf93-e5956fedcdcd None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:20.102245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee515f8-0212-40e0-bf93-e5956fedcdcd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:20.105760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 80/157] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:19 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:20.110928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee515f8-0212-40e0-bf93-e5956fedcdcd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:20.133651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 78/158] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:19 2022] PUT /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d/users/2bca075c6acb413f8998c6a57eca1ac0/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:07:20.227082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:20.232229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee515f8-0212-40e0-bf93-e5956fedcdcd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:20.269549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1a749-b69a-4e23-88c1-8af18b5690ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:20.270238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a749-b69a-4e23-88c1-8af18b5690ed None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:20.270518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a749-b69a-4e23-88c1-8af18b5690ed None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:20.270792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a749-b69a-4e23-88c1-8af18b5690ed None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d/users/2bca075c6acb413f8998c6a57eca1ac0/roles/23a3d7681bf247468a0baa9bfed6f4e9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:20.273079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1a749-b69a-4e23-88c1-8af18b5690ed None admin] RBAC: Authorizing `identity:create_grant(project_id=312738b3aa6f46e2bab330e5c745742d, user_id=2bca075c6acb413f8998c6a57eca1ac0, role_id=23a3d7681bf247468a0baa9bfed6f4e9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:20.276040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1a749-b69a-4e23-88c1-8af18b5690ed None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:20.333514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 81/159] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:20 2022] PUT /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d/users/2bca075c6acb413f8998c6a57eca1ac0/roles/23a3d7681bf247468a0baa9bfed6f4e9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:07:22.015651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5879017-01c6-471f-8619-aa12f7a38803 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.016670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5879017-01c6-471f-8619-aa12f7a38803 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.016958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5879017-01c6-471f-8619-aa12f7a38803 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.018626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 79/160] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:22.023829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a3ad5-623f-49e8-8202-1da03b09539f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.024249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a3ad5-623f-49e8-8202-1da03b09539f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.024442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a3ad5-623f-49e8-8202-1da03b09539f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.047492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3a4a3ad5-623f-49e8-8202-1da03b09539f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:22.049066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3a4a3ad5-623f-49e8-8202-1da03b09539f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:22.071182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6dab-36a2-4820-8099-b515ffed611f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.071495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6dab-36a2-4820-8099-b515ffed611f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.071740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6dab-36a2-4820-8099-b515ffed611f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.073336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 80/161] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:22.078727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609166e-169c-4b86-a80a-9d0d1e551884 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.079011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609166e-169c-4b86-a80a-9d0d1e551884 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.079284 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609166e-169c-4b86-a80a-9d0d1e551884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.102052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d609166e-169c-4b86-a80a-9d0d1e551884 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:22.104298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d609166e-169c-4b86-a80a-9d0d1e551884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:22.108438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:22.138448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 82/162] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:22.180053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 81/163] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:22.368389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc2afb-c0cd-4b8c-9d1a-fa0d98021a00 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.369077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc2afb-c0cd-4b8c-9d1a-fa0d98021a00 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.369580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc2afb-c0cd-4b8c-9d1a-fa0d98021a00 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.371200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 83/164] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:22.376577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b1966-5cb3-44e7-a065-7487b5cbacfe None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.377090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b1966-5cb3-44e7-a065-7487b5cbacfe None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.377558 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b1966-5cb3-44e7-a065-7487b5cbacfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.396347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-572b1966-5cb3-44e7-a065-7487b5cbacfe None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:22.397901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-572b1966-5cb3-44e7-a065-7487b5cbacfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:22.415037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc2afb-c0cd-4b8c-9d1a-fa0d98021a00 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:22.481119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 82/165] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:22.482070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b1966-5cb3-44e7-a065-7487b5cbacfe None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:22.514491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:22.528793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc2afb-c0cd-4b8c-9d1a-fa0d98021a00 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:22.557238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:22.563227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715a82a-64c7-49c5-8417-4a432c4f27f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:22.563853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a82a-64c7-49c5-8417-4a432c4f27f4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.564489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a82a-64c7-49c5-8417-4a432c4f27f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.564489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715a82a-64c7-49c5-8417-4a432c4f27f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.576160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715a82a-64c7-49c5-8417-4a432c4f27f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-2715a82a-64c7-49c5-8417-4a432c4f27f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:22.581097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:22.582132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:22.582932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 84/166] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:22.583590 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab025d2-3397-43a7-80b5-655bde2c0a53 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.584046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab025d2-3397-43a7-80b5-655bde2c0a53 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.584046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab025d2-3397-43a7-80b5-655bde2c0a53 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.585923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 85/167] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:22.589252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab025d2-3397-43a7-80b5-655bde2c0a53 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:22.596393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b1966-5cb3-44e7-a065-7487b5cbacfe None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:22.632963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ab09-2724-4272-9fe7-4fd0ecad36a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:22.633560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ab09-2724-4272-9fe7-4fd0ecad36a0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.633838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ab09-2724-4272-9fe7-4fd0ecad36a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.634117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ab09-2724-4272-9fe7-4fd0ecad36a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.635249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4ab09-2724-4272-9fe7-4fd0ecad36a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:22.636719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4ab09-2724-4272-9fe7-4fd0ecad36a0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:22.638264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 83/168] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:22.639643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284169-a3d0-43c4-a76c-6799bd27d5dd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.639930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284169-a3d0-43c4-a76c-6799bd27d5dd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.640205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c284169-a3d0-43c4-a76c-6799bd27d5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.658829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6c284169-a3d0-43c4-a76c-6799bd27d5dd None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:22.660009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6c284169-a3d0-43c4-a76c-6799bd27d5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:22.701656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab025d2-3397-43a7-80b5-655bde2c0a53 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:22.730470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 84/169] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:22.737622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c284169-a3d0-43c4-a76c-6799bd27d5dd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:22.742675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:22.743337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.743659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.744043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.744831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:22.745195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:22.746650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:22.760115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 86/170] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:22.761220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:22.775092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:22.813078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:22.848443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c284169-a3d0-43c4-a76c-6799bd27d5dd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:22.866674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6abf2a-c1a9-4606-bd9d-6cd9015a8906 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:22.883801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70d12a-b97e-4989-a8c6-c429bd30ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:22.884357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70d12a-b97e-4989-a8c6-c429bd30ab47 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.885245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70d12a-b97e-4989-a8c6-c429bd30ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.885519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70d12a-b97e-4989-a8c6-c429bd30ab47 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.886268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70d12a-b97e-4989-a8c6-c429bd30ab47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:22.887691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70d12a-b97e-4989-a8c6-c429bd30ab47 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:22.899292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd8df9-0a9f-46fb-ba58-74ac90fbf4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:22.899918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8df9-0a9f-46fb-ba58-74ac90fbf4ec None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.900843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8df9-0a9f-46fb-ba58-74ac90fbf4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.901108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8df9-0a9f-46fb-ba58-74ac90fbf4ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.911065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd8df9-0a9f-46fb-ba58-74ac90fbf4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-18fd8df9-0a9f-46fb-ba58-74ac90fbf4ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:22.916323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:22.917196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 87/171] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:07:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:22.918320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd8df9-0a9f-46fb-ba58-74ac90fbf4ec None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:22.986835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.016671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd8df9-0a9f-46fb-ba58-74ac90fbf4ec None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.034005 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 85/172] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:22 2022] POST /identity/v3/users => generated 297 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:23.038166 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70d12a-b97e-4989-a8c6-c429bd30ab47 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.053937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb10176-fde1-4dcb-955c-2d203eb36d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.054668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb10176-fde1-4dcb-955c-2d203eb36d4a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.054870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb10176-fde1-4dcb-955c-2d203eb36d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.055163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb10176-fde1-4dcb-955c-2d203eb36d4a None admin] PATH_INFO: `/v3/users/99f1d2121c7d498685ae4f950852ccd7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.072832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb10176-fde1-4dcb-955c-2d203eb36d4a None admin] RBAC: Authorizing `identity:get_user(user_id=99f1d2121c7d498685ae4f950852ccd7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.074617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb10176-fde1-4dcb-955c-2d203eb36d4a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.078870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 88/173] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:22 2022] GET /identity/v3/users/99f1d2121c7d498685ae4f950852ccd7 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:23.078870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb10176-fde1-4dcb-955c-2d203eb36d4a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.157484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70d12a-b97e-4989-a8c6-c429bd30ab47 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.184552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.195250 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb10176-fde1-4dcb-955c-2d203eb36d4a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.195645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df417458-410f-4348-981a-01407f6adc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.196282 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df417458-410f-4348-981a-01407f6adc58 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.196574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df417458-410f-4348-981a-01407f6adc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.197091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df417458-410f-4348-981a-01407f6adc58 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.199081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df417458-410f-4348-981a-01407f6adc58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:23.199081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df417458-410f-4348-981a-01407f6adc58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.199973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df417458-410f-4348-981a-01407f6adc58 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.212309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 86/174] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:23.212309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df417458-410f-4348-981a-01407f6adc58 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.238040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2725-f1bb-4b13-baf0-0466f30c3834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.238671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2725-f1bb-4b13-baf0-0466f30c3834 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.238950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2725-f1bb-4b13-baf0-0466f30c3834 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.239221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2725-f1bb-4b13-baf0-0466f30c3834 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.253176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c2725-f1bb-4b13-baf0-0466f30c3834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ae4c2725-f1bb-4b13-baf0-0466f30c3834 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:23.261214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:23.262173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.263030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 89/175] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:23.263797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2725-f1bb-4b13-baf0-0466f30c3834 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.335352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df417458-410f-4348-981a-01407f6adc58 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.378594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e55d23-34e2-4571-afc2-f5f558fb829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.379230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e55d23-34e2-4571-afc2-f5f558fb829b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.379507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e55d23-34e2-4571-afc2-f5f558fb829b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.379779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e55d23-34e2-4571-afc2-f5f558fb829b None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.380892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2725-f1bb-4b13-baf0-0466f30c3834 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.383264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e55d23-34e2-4571-afc2-f5f558fb829b None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.386761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e55d23-34e2-4571-afc2-f5f558fb829b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.387713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 87/176] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:23.388307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e55d23-34e2-4571-afc2-f5f558fb829b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.421099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897caa23-4479-46dc-87d1-539422f36cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.421798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897caa23-4479-46dc-87d1-539422f36cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.423032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897caa23-4479-46dc-87d1-539422f36cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.423032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897caa23-4479-46dc-87d1-539422f36cd6 None admin] PATH_INFO: `/v3/users/2bca075c6acb413f8998c6a57eca1ac0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.423510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897caa23-4479-46dc-87d1-539422f36cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=2bca075c6acb413f8998c6a57eca1ac0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.425655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897caa23-4479-46dc-87d1-539422f36cd6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.431025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 90/177] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/users/2bca075c6acb413f8998c6a57eca1ac0 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:23.431025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897caa23-4479-46dc-87d1-539422f36cd6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.457530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.493324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.542359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897caa23-4479-46dc-87d1-539422f36cd6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.565110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e55d23-34e2-4571-afc2-f5f558fb829b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.577506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c4575-df5a-4da1-b5b2-580475e4bde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.578455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c4575-df5a-4da1-b5b2-580475e4bde8 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.578738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c4575-df5a-4da1-b5b2-580475e4bde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.579010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c4575-df5a-4da1-b5b2-580475e4bde8 None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d/users/99f1d2121c7d498685ae4f950852ccd7/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.581025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c4575-df5a-4da1-b5b2-580475e4bde8 None admin] RBAC: Authorizing `identity:create_grant(project_id=312738b3aa6f46e2bab330e5c745742d, user_id=99f1d2121c7d498685ae4f950852ccd7, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.582485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c4575-df5a-4da1-b5b2-580475e4bde8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.604213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.604948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.605293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.605638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.606363 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:23.606749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.608190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.617152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 88/178] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:23.617834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.643321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 91/179] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:23 2022] PUT /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d/users/99f1d2121c7d498685ae4f950852ccd7/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:23.645171 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c4575-df5a-4da1-b5b2-580475e4bde8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.730840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ddffd-3bef-40b8-ae43-24d048a4133c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.768655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.776110 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c4575-df5a-4da1-b5b2-580475e4bde8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.790634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9863e-4e1d-44c7-805d-9d1ebdd967e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.792068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9863e-4e1d-44c7-805d-9d1ebdd967e8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.792365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9863e-4e1d-44c7-805d-9d1ebdd967e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.792637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9863e-4e1d-44c7-805d-9d1ebdd967e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.808449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.840138 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454e18e-4fd5-4150-be96-e35012fe3968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.840816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e18e-4fd5-4150-be96-e35012fe3968 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.841102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e18e-4fd5-4150-be96-e35012fe3968 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.841361 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e18e-4fd5-4150-be96-e35012fe3968 None admin] PATH_INFO: `/v3/users/2bca075c6acb413f8998c6a57eca1ac0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.845048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454e18e-4fd5-4150-be96-e35012fe3968 None admin] RBAC: Authorizing `identity:get_user(user_id=2bca075c6acb413f8998c6a57eca1ac0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.846754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454e18e-4fd5-4150-be96-e35012fe3968 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.848810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 92/180] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/users/2bca075c6acb413f8998c6a57eca1ac0 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:23.855079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454e18e-4fd5-4150-be96-e35012fe3968 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.872933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9863e-4e1d-44c7-805d-9d1ebdd967e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.874422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9863e-4e1d-44c7-805d-9d1ebdd967e8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.989091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 89/181] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:23.995575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9863e-4e1d-44c7-805d-9d1ebdd967e8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:24.020650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:24.024179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454e18e-4fd5-4150-be96-e35012fe3968 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:24.063964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25a5ea-508c-4ed2-a138-c638f612211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:24.064619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a5ea-508c-4ed2-a138-c638f612211c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:24.064886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a5ea-508c-4ed2-a138-c638f612211c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:24.065161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a5ea-508c-4ed2-a138-c638f612211c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:24.143538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9863e-4e1d-44c7-805d-9d1ebdd967e8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:24.160213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:24.193359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25a5ea-508c-4ed2-a138-c638f612211c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:24.194902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25a5ea-508c-4ed2-a138-c638f612211c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:24.220440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6016a-1dda-4ef5-a6e9-2df2f061bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:24.221077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6016a-1dda-4ef5-a6e9-2df2f061bcac None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:24.221358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6016a-1dda-4ef5-a6e9-2df2f061bcac None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:24.221630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6016a-1dda-4ef5-a6e9-2df2f061bcac None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:24.224086 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6016a-1dda-4ef5-a6e9-2df2f061bcac None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:24.225603 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6016a-1dda-4ef5-a6e9-2df2f061bcac None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:24.227551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 90/182] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:24.233292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6016a-1dda-4ef5-a6e9-2df2f061bcac None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:24.319312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 93/183] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:24.326089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25a5ea-508c-4ed2-a138-c638f612211c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:24.364628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:24.376768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6016a-1dda-4ef5-a6e9-2df2f061bcac None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:24.411335 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:24.411947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:24.412233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:24.412541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:24.413268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=312738b3aa6f46e2bab330e5c745742d, user.id=2bca075c6acb413f8998c6a57eca1ac0) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:24.413579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:24.415016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:24.421122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 91/184] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:24 2022] GET /identity/v3/role_assignments?user.id=2bca075c6acb413f8998c6a57eca1ac0&scope.project.id=312738b3aa6f46e2bab330e5c745742d&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:24.470505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:24.498341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25a5ea-508c-4ed2-a138-c638f612211c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:24.546424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34751a89-8ae6-4f5a-9f8c-4d2086d0365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:24.547059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34751a89-8ae6-4f5a-9f8c-4d2086d0365a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:24.547334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34751a89-8ae6-4f5a-9f8c-4d2086d0365a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:24.547605 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34751a89-8ae6-4f5a-9f8c-4d2086d0365a None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:24.548732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34751a89-8ae6-4f5a-9f8c-4d2086d0365a None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:24.550170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34751a89-8ae6-4f5a-9f8c-4d2086d0365a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:24.551911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 94/185] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:24 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:24.558719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:24.707115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2fb19-071b-40c4-a3bc-3ecb8385a6f2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:24.747721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4be17-93de-4a0a-a130-6f5cf1363502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:24.748717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4be17-93de-4a0a-a130-6f5cf1363502 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:24.749223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4be17-93de-4a0a-a130-6f5cf1363502 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:24.749733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4be17-93de-4a0a-a130-6f5cf1363502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:24.750669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4be17-93de-4a0a-a130-6f5cf1363502 None admin] RBAC: Adding query filter params (role.id=23a3d7681bf247468a0baa9bfed6f4e9, scope.project.id=312738b3aa6f46e2bab330e5c745742d, user.id=2bca075c6acb413f8998c6a57eca1ac0) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:24.751192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4be17-93de-4a0a-a130-6f5cf1363502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:24.752853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4be17-93de-4a0a-a130-6f5cf1363502 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:24.760701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 92/186] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:24 2022] GET /identity/v3/role_assignments?user.id=2bca075c6acb413f8998c6a57eca1ac0&scope.project.id=312738b3aa6f46e2bab330e5c745742d&role.id=23a3d7681bf247468a0baa9bfed6f4e9 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:25.551031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a94cc-b19c-4bb7-9311-c348a63afd2d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:25.551553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a94cc-b19c-4bb7-9311-c348a63afd2d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:25.551553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a94cc-b19c-4bb7-9311-c348a63afd2d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:25.553113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 95/187] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:25.558364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6618d11-6a67-41db-9011-d30fdd2a8fb8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:25.558886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6618d11-6a67-41db-9011-d30fdd2a8fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:25.558960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6618d11-6a67-41db-9011-d30fdd2a8fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:25.578141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f6618d11-6a67-41db-9011-d30fdd2a8fb8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:25.579352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f6618d11-6a67-41db-9011-d30fdd2a8fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:25.588577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:25.652656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 93/188] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:25.693641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78348c95-0eb8-4cda-ae62-5acf11f7a615 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:25.693936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78348c95-0eb8-4cda-ae62-5acf11f7a615 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:25.694205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78348c95-0eb8-4cda-ae62-5acf11f7a615 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:25.695680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 96/189] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:25.701371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca4f85-7928-493e-91d5-2cf36a659028 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:25.701661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca4f85-7928-493e-91d5-2cf36a659028 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:25.701937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca4f85-7928-493e-91d5-2cf36a659028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:25.720759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9aca4f85-7928-493e-91d5-2cf36a659028 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:25.721945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9aca4f85-7928-493e-91d5-2cf36a659028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:25.794513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 94/190] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:25.918240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78348c95-0eb8-4cda-ae62-5acf11f7a615 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.020318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78348c95-0eb8-4cda-ae62-5acf11f7a615 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.058985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14fd8f-4ed1-4a6c-ac2a-bc2f9c54579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.059629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14fd8f-4ed1-4a6c-ac2a-bc2f9c54579f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.059908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14fd8f-4ed1-4a6c-ac2a-bc2f9c54579f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.060177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14fd8f-4ed1-4a6c-ac2a-bc2f9c54579f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.066243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca4f85-7928-493e-91d5-2cf36a659028 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.071391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14fd8f-4ed1-4a6c-ac2a-bc2f9c54579f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-cd14fd8f-4ed1-4a6c-ac2a-bc2f9c54579f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:26.076796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:26.077864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:26.078770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 97/191] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:26.083557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14fd8f-4ed1-4a6c-ac2a-bc2f9c54579f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.103514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:26.152887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:26.182235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca4f85-7928-493e-91d5-2cf36a659028 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.189750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14fd8f-4ed1-4a6c-ac2a-bc2f9c54579f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.216586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744f441-8a56-434a-a57a-91438782ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.217211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744f441-8a56-434a-a57a-91438782ac5a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.217490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744f441-8a56-434a-a57a-91438782ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.217764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744f441-8a56-434a-a57a-91438782ac5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.219107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744f441-8a56-434a-a57a-91438782ac5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:26.220567 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744f441-8a56-434a-a57a-91438782ac5a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:26.222121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 95/192] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:26.222593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb60bc4-b5cb-427c-831e-6c6d2c7e71c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.223200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb60bc4-b5cb-427c-831e-6c6d2c7e71c8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.223453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb60bc4-b5cb-427c-831e-6c6d2c7e71c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.223722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb60bc4-b5cb-427c-831e-6c6d2c7e71c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.224413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb60bc4-b5cb-427c-831e-6c6d2c7e71c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:26.224711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb60bc4-b5cb-427c-831e-6c6d2c7e71c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:26.225801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674fb3b-2d46-4872-81b4-89d65751c70e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.226052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb60bc4-b5cb-427c-831e-6c6d2c7e71c8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:26.226156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674fb3b-2d46-4872-81b4-89d65751c70e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.226340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674fb3b-2d46-4872-81b4-89d65751c70e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.227886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 96/193] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:26.228617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674fb3b-2d46-4872-81b4-89d65751c70e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.238597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 98/194] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:26.238597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333cce-da79-4368-ba7a-40fc7187337a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.238597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333cce-da79-4368-ba7a-40fc7187337a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.238597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7333cce-da79-4368-ba7a-40fc7187337a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.253152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c7333cce-da79-4368-ba7a-40fc7187337a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:26.254319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c7333cce-da79-4368-ba7a-40fc7187337a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:26.318620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 99/195] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:26.319446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7333cce-da79-4368-ba7a-40fc7187337a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.322883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:26.331241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674fb3b-2d46-4872-81b4-89d65751c70e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.370628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059623bf-f0d0-43f2-84a4-54b879bd9c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.371227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059623bf-f0d0-43f2-84a4-54b879bd9c49 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.371510 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059623bf-f0d0-43f2-84a4-54b879bd9c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.371781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059623bf-f0d0-43f2-84a4-54b879bd9c49 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.372459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059623bf-f0d0-43f2-84a4-54b879bd9c49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:26.373870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059623bf-f0d0-43f2-84a4-54b879bd9c49 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:26.386303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:26.419317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7333cce-da79-4368-ba7a-40fc7187337a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.453233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d26ed-7621-41f3-8a58-926cdb96f6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.453791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d26ed-7621-41f3-8a58-926cdb96f6ef None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.454070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d26ed-7621-41f3-8a58-926cdb96f6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.454356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d26ed-7621-41f3-8a58-926cdb96f6ef None admin] PATH_INFO: `/v3/users/97f3cd7119ba4c5891f950e21e3c6975` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.467280 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d26ed-7621-41f3-8a58-926cdb96f6ef None admin] RBAC: Authorizing `identity:get_user(user_id=97f3cd7119ba4c5891f950e21e3c6975)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:26.469014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d26ed-7621-41f3-8a58-926cdb96f6ef None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:26.470601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 100/196] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:26 2022] GET /identity/v3/users/97f3cd7119ba4c5891f950e21e3c6975 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:26.476191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d26ed-7621-41f3-8a58-926cdb96f6ef None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.490013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 97/197] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:26 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:26.543238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:26.584945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d26ed-7621-41f3-8a58-926cdb96f6ef None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.590487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059623bf-f0d0-43f2-84a4-54b879bd9c49 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.621742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538de4e-0aa9-40bd-b9bf-de7339a45f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.622380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538de4e-0aa9-40bd-b9bf-de7339a45f6b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.622658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538de4e-0aa9-40bd-b9bf-de7339a45f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.622927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538de4e-0aa9-40bd-b9bf-de7339a45f6b None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.624065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538de4e-0aa9-40bd-b9bf-de7339a45f6b None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:26.625562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538de4e-0aa9-40bd-b9bf-de7339a45f6b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:26.627217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 101/198] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:26 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:26.633268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538de4e-0aa9-40bd-b9bf-de7339a45f6b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.714171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059623bf-f0d0-43f2-84a4-54b879bd9c49 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.754647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538de4e-0aa9-40bd-b9bf-de7339a45f6b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.760363 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c731-31d2-41c1-8dd4-ad080b57d0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.760999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c731-31d2-41c1-8dd4-ad080b57d0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.761277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c731-31d2-41c1-8dd4-ad080b57d0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.761550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c731-31d2-41c1-8dd4-ad080b57d0f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.775032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0c731-31d2-41c1-8dd4-ad080b57d0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-afc0c731-31d2-41c1-8dd4-ad080b57d0f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:26.781014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:26.781929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:26.782753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 98/199] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:26.789325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c731-31d2-41c1-8dd4-ad080b57d0f8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.795768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:26.802587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d4371-02e6-46f6-b153-6d8784ab1d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.803835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4371-02e6-46f6-b153-6d8784ab1d43 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.804370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4371-02e6-46f6-b153-6d8784ab1d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.805076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4371-02e6-46f6-b153-6d8784ab1d43 None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d/users/97f3cd7119ba4c5891f950e21e3c6975/roles/ab3629eff4074dd29639a73114e696d4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.808982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d4371-02e6-46f6-b153-6d8784ab1d43 None admin] RBAC: Authorizing `identity:create_grant(project_id=312738b3aa6f46e2bab330e5c745742d, user_id=97f3cd7119ba4c5891f950e21e3c6975, role_id=ab3629eff4074dd29639a73114e696d4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:26.810710 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d4371-02e6-46f6-b153-6d8784ab1d43 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:26.854570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:26.857922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 102/200] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:26 2022] PUT /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d/users/97f3cd7119ba4c5891f950e21e3c6975/roles/ab3629eff4074dd29639a73114e696d4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:07:26.907620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c731-31d2-41c1-8dd4-ad080b57d0f8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.949443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.950049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.950508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.950789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.951467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:26.951840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:26.953317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:26.963469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 99/201] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:26.969987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d4371-02e6-46f6-b153-6d8784ab1d43 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:27.085391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d4371-02e6-46f6-b153-6d8784ab1d43 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:27.123260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83529-4b4d-4025-b7d5-7f6221c4a7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:27.124301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83529-4b4d-4025-b7d5-7f6221c4a7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:27.124796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83529-4b4d-4025-b7d5-7f6221c4a7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:27.125267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83529-4b4d-4025-b7d5-7f6221c4a7f4 None admin] PATH_INFO: `/v3/users/99f1d2121c7d498685ae4f950852ccd7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:27.126596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd83529-4b4d-4025-b7d5-7f6221c4a7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=99f1d2121c7d498685ae4f950852ccd7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:27.128226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd83529-4b4d-4025-b7d5-7f6221c4a7f4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:27.130065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 103/202] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:26 2022] GET /identity/v3/users/99f1d2121c7d498685ae4f950852ccd7 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:27.140308 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:27.240975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2b3f1-704d-4cf3-8d36-e3823bcdc84a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:27.258787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:27.275885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfae18-9de0-4877-bc5d-ba20483a3ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:27.276526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfae18-9de0-4877-bc5d-ba20483a3ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:27.276803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfae18-9de0-4877-bc5d-ba20483a3ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:27.277079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfae18-9de0-4877-bc5d-ba20483a3ce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:27.351286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfae18-9de0-4877-bc5d-ba20483a3ce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:27.352782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfae18-9de0-4877-bc5d-ba20483a3ce8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:27.454837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 100/203] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:27.460582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83529-4b4d-4025-b7d5-7f6221c4a7f4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:27.521733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:27.564488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83529-4b4d-4025-b7d5-7f6221c4a7f4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:27.599230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c9939-abd2-48c7-a2bf-f734dee6bb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:27.599852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c9939-abd2-48c7-a2bf-f734dee6bb91 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:27.600114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c9939-abd2-48c7-a2bf-f734dee6bb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:27.600412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c9939-abd2-48c7-a2bf-f734dee6bb91 None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:27.601516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c9939-abd2-48c7-a2bf-f734dee6bb91 None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:27.602992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c9939-abd2-48c7-a2bf-f734dee6bb91 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:27.605266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 104/204] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:27 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:27.611296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfae18-9de0-4877-bc5d-ba20483a3ce8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:27.630959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:27.720805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfae18-9de0-4877-bc5d-ba20483a3ce8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:27.755937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1a367-bab3-4b67-94bc-8328ff6c12b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:27.756943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1a367-bab3-4b67-94bc-8328ff6c12b2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:27.757419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1a367-bab3-4b67-94bc-8328ff6c12b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:27.757861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1a367-bab3-4b67-94bc-8328ff6c12b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:27.758751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1a367-bab3-4b67-94bc-8328ff6c12b2 None admin] RBAC: Adding query filter params (role.id=eb33c965778e405b95ebf794f31e329a, scope.project.id=312738b3aa6f46e2bab330e5c745742d, user.id=99f1d2121c7d498685ae4f950852ccd7) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:27.759228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1a367-bab3-4b67-94bc-8328ff6c12b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:27.761398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1a367-bab3-4b67-94bc-8328ff6c12b2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:27.772462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 101/205] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:27 2022] GET /identity/v3/role_assignments?user.id=99f1d2121c7d498685ae4f950852ccd7&scope.project.id=312738b3aa6f46e2bab330e5c745742d&role.id=eb33c965778e405b95ebf794f31e329a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:28.804850 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdaf10-fa66-45e3-a25e-564bed66cdcd None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:28.805415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdaf10-fa66-45e3-a25e-564bed66cdcd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:28.805415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdaf10-fa66-45e3-a25e-564bed66cdcd None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:28.807333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 105/206] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:28.813393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd89b2-a817-4f6e-a8cc-d98937f9ed0e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:28.813902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd89b2-a817-4f6e-a8cc-d98937f9ed0e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:28.814001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd89b2-a817-4f6e-a8cc-d98937f9ed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:28.833055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-11fd89b2-a817-4f6e-a8cc-d98937f9ed0e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:28.834264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-11fd89b2-a817-4f6e-a8cc-d98937f9ed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:28.900746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 102/207] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:28.937873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3b19b-cf75-4495-bc9c-f4614854d295 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:28.938156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3b19b-cf75-4495-bc9c-f4614854d295 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:28.938615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3b19b-cf75-4495-bc9c-f4614854d295 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:28.940112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 106/208] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:28.945393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ccc4-d591-47ec-9237-70fa724ca91e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:28.945848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ccc4-d591-47ec-9237-70fa724ca91e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:28.946266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ccc4-d591-47ec-9237-70fa724ca91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:28.964440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a4f2ccc4-d591-47ec-9237-70fa724ca91e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:28.965801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a4f2ccc4-d591-47ec-9237-70fa724ca91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:28.974495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.039271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 103/209] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:29.097996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bdd4f-8be4-45d2-a3c7-8356f8819983 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.098288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bdd4f-8be4-45d2-a3c7-8356f8819983 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.098561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bdd4f-8be4-45d2-a3c7-8356f8819983 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.099899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 107/210] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:29.105696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec689e-4247-4496-addb-5e72356c1175 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.105989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec689e-4247-4496-addb-5e72356c1175 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.106294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec689e-4247-4496-addb-5e72356c1175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.124347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9eec689e-4247-4496-addb-5e72356c1175 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:29.125525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9eec689e-4247-4496-addb-5e72356c1175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:29.167060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bdd4f-8be4-45d2-a3c7-8356f8819983 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.194447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 104/211] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:29.266638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bdd4f-8be4-45d2-a3c7-8356f8819983 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.292206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.300518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6976d-c50e-439b-aecb-4d3601c53efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.302985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6976d-c50e-439b-aecb-4d3601c53efc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.302985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6976d-c50e-439b-aecb-4d3601c53efc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.302985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6976d-c50e-439b-aecb-4d3601c53efc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.311852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6976d-c50e-439b-aecb-4d3601c53efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:29.311994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec689e-4247-4496-addb-5e72356c1175 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-eec6976d-c50e-439b-aecb-4d3601c53efc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:29.316708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:29.317606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:29.318420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 108/212] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:29.323423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6976d-c50e-439b-aecb-4d3601c53efc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.395188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.426780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6976d-c50e-439b-aecb-4d3601c53efc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.427602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec689e-4247-4496-addb-5e72356c1175 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.461333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.461934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.462211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.462505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.463697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:29.463979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:29.465372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:29.468727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adc755-80f7-4bb9-b6a6-d8b5329a2918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.468727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adc755-80f7-4bb9-b6a6-d8b5329a2918 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.468727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adc755-80f7-4bb9-b6a6-d8b5329a2918 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.468727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adc755-80f7-4bb9-b6a6-d8b5329a2918 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.469001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adc755-80f7-4bb9-b6a6-d8b5329a2918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:29.469915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adc755-80f7-4bb9-b6a6-d8b5329a2918 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:29.471515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 105/213] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:29.472300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adc755-80f7-4bb9-b6a6-d8b5329a2918 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.475358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 109/214] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:29.477124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.525966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.572822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adc755-80f7-4bb9-b6a6-d8b5329a2918 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.576616 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fa951-d6ae-4fcb-b81e-41b8f5d3683c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.607140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66842-e95f-4828-9ad8-d1ca360e426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.607766 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66842-e95f-4828-9ad8-d1ca360e426d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.608045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66842-e95f-4828-9ad8-d1ca360e426d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.608344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66842-e95f-4828-9ad8-d1ca360e426d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.609155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e4004-b99e-4af5-8246-c00fd697feee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.609743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e4004-b99e-4af5-8246-c00fd697feee None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.610018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e4004-b99e-4af5-8246-c00fd697feee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.610280 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e4004-b99e-4af5-8246-c00fd697feee None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.610921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e4004-b99e-4af5-8246-c00fd697feee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:29.612436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e4004-b99e-4af5-8246-c00fd697feee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:29.615616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.618989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e66842-e95f-4828-9ad8-d1ca360e426d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-77e66842-e95f-4828-9ad8-d1ca360e426d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:29.623742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:29.624703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:29.625537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 106/215] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:29.626201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66842-e95f-4828-9ad8-d1ca360e426d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.708396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 110/216] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:29 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:29.708396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e4004-b99e-4af5-8246-c00fd697feee None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.733141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66842-e95f-4828-9ad8-d1ca360e426d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.771917 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3643c84-c821-4e54-adda-bd1769885642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.772590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3643c84-c821-4e54-adda-bd1769885642 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.772874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3643c84-c821-4e54-adda-bd1769885642 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.773149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3643c84-c821-4e54-adda-bd1769885642 None admin] PATH_INFO: `/v3/users/81b093e60c0446faa63a11e7c8444f25` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.789912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3643c84-c821-4e54-adda-bd1769885642 None admin] RBAC: Authorizing `identity:get_user(user_id=81b093e60c0446faa63a11e7c8444f25)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:29.791703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3643c84-c821-4e54-adda-bd1769885642 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:29.793645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 107/217] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/users/81b093e60c0446faa63a11e7c8444f25 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:29.799648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3643c84-c821-4e54-adda-bd1769885642 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.823670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e4004-b99e-4af5-8246-c00fd697feee None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.826279 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.860881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.861503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.861778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.862052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.862721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:29.863023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:29.864454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:29.870029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.873939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 111/218] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:29.881432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.916614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3643c84-c821-4e54-adda-bd1769885642 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.961069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.961602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.961878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.962149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.962915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] RBAC: Adding query filter params (role.id=eb33c965778e405b95ebf794f31e329a, user.id=81b093e60c0446faa63a11e7c8444f25, scope.system=all) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:29.963221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:29.964684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:29.971980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 108/219] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/role_assignments?user.id=81b093e60c0446faa63a11e7c8444f25&scope.system=all&role.id=eb33c965778e405b95ebf794f31e329a => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:30.001623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e847-c70a-4052-8aa5-a1aaf94ec7c0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:30.044064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1989db-dfae-4334-98aa-d52801ebed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:30.045159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1989db-dfae-4334-98aa-d52801ebed59 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:30.045440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1989db-dfae-4334-98aa-d52801ebed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:30.045710 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1989db-dfae-4334-98aa-d52801ebed59 None admin] PATH_INFO: `/v3/users/97f3cd7119ba4c5891f950e21e3c6975` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:30.046903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1989db-dfae-4334-98aa-d52801ebed59 None admin] RBAC: Authorizing `identity:get_user(user_id=97f3cd7119ba4c5891f950e21e3c6975)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:30.048470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1989db-dfae-4334-98aa-d52801ebed59 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:30.050344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 112/220] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/users/97f3cd7119ba4c5891f950e21e3c6975 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:30.056067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:30.174557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310456d2-e861-4d9a-9dd7-616f34d0bd11 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:30.198690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:30.210375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d7f7d-14e2-42c7-8a1d-2ab5ab63557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:30.210987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d7f7d-14e2-42c7-8a1d-2ab5ab63557d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:30.220646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d7f7d-14e2-42c7-8a1d-2ab5ab63557d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:30.220646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d7f7d-14e2-42c7-8a1d-2ab5ab63557d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:30.281619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d7f7d-14e2-42c7-8a1d-2ab5ab63557d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:30.283158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d7f7d-14e2-42c7-8a1d-2ab5ab63557d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:30.386345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 109/221] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:30.392040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1989db-dfae-4334-98aa-d52801ebed59 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:30.427789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:30.544519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1989db-dfae-4334-98aa-d52801ebed59 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:30.579658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039acb8-eae2-464e-b317-d5de3c0fed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:30.580309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039acb8-eae2-464e-b317-d5de3c0fed58 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:30.580588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039acb8-eae2-464e-b317-d5de3c0fed58 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:30.580862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039acb8-eae2-464e-b317-d5de3c0fed58 None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:30.581957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039acb8-eae2-464e-b317-d5de3c0fed58 None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:30.583391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039acb8-eae2-464e-b317-d5de3c0fed58 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:30.585104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 113/222] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:30 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:30.591726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d7f7d-14e2-42c7-8a1d-2ab5ab63557d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:30.616955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:30.696288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d7f7d-14e2-42c7-8a1d-2ab5ab63557d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:30.732127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf521c4a-59f9-4a76-9015-a1d21e4d0182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:30.732774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf521c4a-59f9-4a76-9015-a1d21e4d0182 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:30.733051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf521c4a-59f9-4a76-9015-a1d21e4d0182 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:30.733320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf521c4a-59f9-4a76-9015-a1d21e4d0182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:30.734061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf521c4a-59f9-4a76-9015-a1d21e4d0182 None admin] RBAC: Adding query filter params (role.id=ab3629eff4074dd29639a73114e696d4, scope.project.id=312738b3aa6f46e2bab330e5c745742d, user.id=97f3cd7119ba4c5891f950e21e3c6975) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:30.734371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf521c4a-59f9-4a76-9015-a1d21e4d0182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:30.735921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf521c4a-59f9-4a76-9015-a1d21e4d0182 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:30.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 110/223] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:07:30 2022] GET /identity/v3/role_assignments?user.id=97f3cd7119ba4c5891f950e21e3c6975&scope.project.id=312738b3aa6f46e2bab330e5c745742d&role.id=ab3629eff4074dd29639a73114e696d4 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:32.016509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7dc6f-0bb9-48ad-821d-ac5b0601c8a9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.017072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7dc6f-0bb9-48ad-821d-ac5b0601c8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.017072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7dc6f-0bb9-48ad-821d-ac5b0601c8a9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.018592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 114/224] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:32.024197 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa15c4-7680-4acd-be20-16354f5f5a84 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.024753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa15c4-7680-4acd-be20-16354f5f5a84 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.024753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa15c4-7680-4acd-be20-16354f5f5a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.045421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-eefa15c4-7680-4acd-be20-16354f5f5a84 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:32.046646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eefa15c4-7680-4acd-be20-16354f5f5a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:32.096532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8437e-a64d-4e71-9fa5-6bc89a32ebfe None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.096815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8437e-a64d-4e71-9fa5-6bc89a32ebfe None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.097084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8437e-a64d-4e71-9fa5-6bc89a32ebfe None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.098540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 115/225] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:32.103373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfb6f5-dc8f-46f6-b089-645cb40c9706 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.103654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfb6f5-dc8f-46f6-b089-645cb40c9706 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.103929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfb6f5-dc8f-46f6-b089-645cb40c9706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.122391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 111/226] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:32.122561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f3bfb6f5-dc8f-46f6-b089-645cb40c9706 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:32.123691 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f3bfb6f5-dc8f-46f6-b089-645cb40c9706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:32.146260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f1025-eb1d-473d-a052-6af35d72d324 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.146563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f1025-eb1d-473d-a052-6af35d72d324 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.146817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f1025-eb1d-473d-a052-6af35d72d324 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.148288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 112/227] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:32.153093 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0be2-06b5-4bcb-ab6e-1fc273891662 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.153371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0be2-06b5-4bcb-ab6e-1fc273891662 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.154169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f0be2-06b5-4bcb-ab6e-1fc273891662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.173003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-670f0be2-06b5-4bcb-ab6e-1fc273891662 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:32.174500 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-670f0be2-06b5-4bcb-ab6e-1fc273891662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:32.185717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:32.200652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 116/228] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:32.240508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 113/229] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:32.389591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfb6f5-dc8f-46f6-b089-645cb40c9706 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.420643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:32.475280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f0be2-06b5-4bcb-ab6e-1fc273891662 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.489770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfb6f5-dc8f-46f6-b089-645cb40c9706 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:32.525264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fa133-438e-4294-a586-a7554d129676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:32.525886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fa133-438e-4294-a586-a7554d129676 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.526160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fa133-438e-4294-a586-a7554d129676 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.526426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fa133-438e-4294-a586-a7554d129676 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.536800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fa133-438e-4294-a586-a7554d129676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7d2fa133-438e-4294-a586-a7554d129676 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:32.542126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:32.543221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 117/230] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:32.544063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fa133-438e-4294-a586-a7554d129676 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.578424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f0be2-06b5-4bcb-ab6e-1fc273891662 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:32.611835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f2124-8fae-4be2-983c-078c7dfbcf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:32.612476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f2124-8fae-4be2-983c-078c7dfbcf78 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.612752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f2124-8fae-4be2-983c-078c7dfbcf78 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.613020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f2124-8fae-4be2-983c-078c7dfbcf78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.614095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f2124-8fae-4be2-983c-078c7dfbcf78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:32.615509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f2124-8fae-4be2-983c-078c7dfbcf78 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:32.617081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 114/231] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:32.619927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f2124-8fae-4be2-983c-078c7dfbcf78 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.633581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:32.635794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:32.645236 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fa133-438e-4294-a586-a7554d129676 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:32.674765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7d655-4865-49c1-a458-d09d03225976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:32.675383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7d655-4865-49c1-a458-d09d03225976 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.675663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7d655-4865-49c1-a458-d09d03225976 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.675932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7d655-4865-49c1-a458-d09d03225976 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.686657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7d655-4865-49c1-a458-d09d03225976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5eb7d655-4865-49c1-a458-d09d03225976 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:32.691414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:32.692515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 118/232] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:32.693357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7d655-4865-49c1-a458-d09d03225976 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.719371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f2124-8fae-4be2-983c-078c7dfbcf78 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:32.754883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:32.755476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.755751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.756017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.756707 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:32.757000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:32.758354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:32.766117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 115/233] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:32.766901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.789974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7d655-4865-49c1-a458-d09d03225976 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:32.822039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ac79a-b6f6-49a5-8f7a-d13f25afdcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:32.822633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ac79a-b6f6-49a5-8f7a-d13f25afdcbb None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.822908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ac79a-b6f6-49a5-8f7a-d13f25afdcbb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.823179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ac79a-b6f6-49a5-8f7a-d13f25afdcbb None admin] PATH_INFO: `/v3/groups` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.823788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ac79a-b6f6-49a5-8f7a-d13f25afdcbb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:32.825234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ac79a-b6f6-49a5-8f7a-d13f25afdcbb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:32.844209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:32.866304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c739570-6804-4d15-809d-abb944367d95 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:32.897234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:32.897764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.898039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.898302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.898945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:32.899258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:32.901231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:32.906736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 119/234] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Oct 24 13:07:32 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:32.907527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ac79a-b6f6-49a5-8f7a-d13f25afdcbb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.913874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 116/235] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:32.914738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.924869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:33.018335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ac79a-b6f6-49a5-8f7a-d13f25afdcbb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:33.029321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc5b37-a1f1-4576-abb7-9d01db3fe76b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:33.056515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d80f6-e3b8-445a-908a-98697b919523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:33.057174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d80f6-e3b8-445a-908a-98697b919523 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:33.057434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d80f6-e3b8-445a-908a-98697b919523 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:33.057698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d80f6-e3b8-445a-908a-98697b919523 None admin] PATH_INFO: `/v3/users/4529e93db3b547c98140ff9a504fb311` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:33.066791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48746610-bc66-45ba-bf5f-2fdb39dc920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:33.067432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48746610-bc66-45ba-bf5f-2fdb39dc920c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:33.067716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48746610-bc66-45ba-bf5f-2fdb39dc920c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:33.068007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48746610-bc66-45ba-bf5f-2fdb39dc920c None admin] PATH_INFO: `/v3/users/81b093e60c0446faa63a11e7c8444f25` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:33.069220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48746610-bc66-45ba-bf5f-2fdb39dc920c None admin] RBAC: Authorizing `identity:get_user(user_id=81b093e60c0446faa63a11e7c8444f25)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:33.070763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48746610-bc66-45ba-bf5f-2fdb39dc920c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:33.072411 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 117/236] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:32 2022] GET /identity/v3/users/81b093e60c0446faa63a11e7c8444f25 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:33.072992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d80f6-e3b8-445a-908a-98697b919523 None admin] RBAC: Authorizing `identity:get_user(user_id=4529e93db3b547c98140ff9a504fb311)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:33.074531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d80f6-e3b8-445a-908a-98697b919523 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:33.076323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 120/237] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:32 2022] GET /identity/v3/users/4529e93db3b547c98140ff9a504fb311 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:33.083276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48746610-bc66-45ba-bf5f-2fdb39dc920c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:33.084384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d80f6-e3b8-445a-908a-98697b919523 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:33.094610 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:33.166887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:33.195971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48746610-bc66-45ba-bf5f-2fdb39dc920c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:33.197745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d80f6-e3b8-445a-908a-98697b919523 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:33.230583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730236d-7a1c-48f8-9c1b-0a8cb17057b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:33.231228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730236d-7a1c-48f8-9c1b-0a8cb17057b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:33.231503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730236d-7a1c-48f8-9c1b-0a8cb17057b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:33.231773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730236d-7a1c-48f8-9c1b-0a8cb17057b9 None admin] PATH_INFO: `/v3/system/users/81b093e60c0446faa63a11e7c8444f25/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:33.233196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730236d-7a1c-48f8-9c1b-0a8cb17057b9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=81b093e60c0446faa63a11e7c8444f25, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:33.234831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730236d-7a1c-48f8-9c1b-0a8cb17057b9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:33.240155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da024df-090c-47ae-835c-e8d7604405f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:33.240811 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da024df-090c-47ae-835c-e8d7604405f5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:33.241089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da024df-090c-47ae-835c-e8d7604405f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:33.241357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da024df-090c-47ae-835c-e8d7604405f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:33.242119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da024df-090c-47ae-835c-e8d7604405f5 None admin] RBAC: Adding query filter params (role.id=ab3629eff4074dd29639a73114e696d4, user.id=4529e93db3b547c98140ff9a504fb311, scope.system=all) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:33.242422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da024df-090c-47ae-835c-e8d7604405f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:33.244107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da024df-090c-47ae-835c-e8d7604405f5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:33.249832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 121/238] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Oct 24 13:07:33 2022] GET /identity/v3/role_assignments?user.id=4529e93db3b547c98140ff9a504fb311&scope.system=all&role.id=ab3629eff4074dd29639a73114e696d4 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:33.326268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 118/239] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Oct 24 13:07:33 2022] PUT /identity/v3/system/users/81b093e60c0446faa63a11e7c8444f25/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:35.091076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1eee7-a279-457e-8e96-af86a4df7992 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.091580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1eee7-a279-457e-8e96-af86a4df7992 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.091580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1eee7-a279-457e-8e96-af86a4df7992 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.093142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 122/240] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:35.098734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cbebe-282f-4a19-9115-b1ba20993b61 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.099116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cbebe-282f-4a19-9115-b1ba20993b61 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.099276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cbebe-282f-4a19-9115-b1ba20993b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.119575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c43cbebe-282f-4a19-9115-b1ba20993b61 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:35.120787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c43cbebe-282f-4a19-9115-b1ba20993b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:35.184908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:35.189634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 119/241] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:35.386877 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994574-14fb-4009-be8c-28eae5ad6f04 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.387040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994574-14fb-4009-be8c-28eae5ad6f04 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.387321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994574-14fb-4009-be8c-28eae5ad6f04 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.388853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 123/242] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:35.393732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddc987-471d-4bc8-8b8a-d2c880a04c8e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.394019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddc987-471d-4bc8-8b8a-d2c880a04c8e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.394301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddc987-471d-4bc8-8b8a-d2c880a04c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.412446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-99ddc987-471d-4bc8-8b8a-d2c880a04c8e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:35.413639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-99ddc987-471d-4bc8-8b8a-d2c880a04c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:35.450019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ddfa1-4a4c-4bf6-9b29-42f73796f723 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.450315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ddfa1-4a4c-4bf6-9b29-42f73796f723 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.450630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ddfa1-4a4c-4bf6-9b29-42f73796f723 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.451923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 124/243] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:07:35.456989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01deca9-a84b-4773-ae6a-e6022053d116 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.457265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01deca9-a84b-4773-ae6a-e6022053d116 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.457539 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01deca9-a84b-4773-ae6a-e6022053d116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.475866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d01deca9-a84b-4773-ae6a-e6022053d116 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:35.477065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d01deca9-a84b-4773-ae6a-e6022053d116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:35.555557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 120/244] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:35.557111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddc987-471d-4bc8-8b8a-d2c880a04c8e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:35.659472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 125/245] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:35.728434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddc987-471d-4bc8-8b8a-d2c880a04c8e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:35.756337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:35.762397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239bfd0-1adb-4a1c-bec9-64f16f3052a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:35.763019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239bfd0-1adb-4a1c-bec9-64f16f3052a5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.763293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239bfd0-1adb-4a1c-bec9-64f16f3052a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.763562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239bfd0-1adb-4a1c-bec9-64f16f3052a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.764664 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239bfd0-1adb-4a1c-bec9-64f16f3052a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:35.766098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239bfd0-1adb-4a1c-bec9-64f16f3052a5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:35.767805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 121/246] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:35.773114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01deca9-a84b-4773-ae6a-e6022053d116 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:35.825888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239bfd0-1adb-4a1c-bec9-64f16f3052a5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:35.839422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:35.877339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01deca9-a84b-4773-ae6a-e6022053d116 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:35.911234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9dce7-a86e-4b10-a66a-ff449c72c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:35.911906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9dce7-a86e-4b10-a66a-ff449c72c3a7 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.912015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9dce7-a86e-4b10-a66a-ff449c72c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.912308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9dce7-a86e-4b10-a66a-ff449c72c3a7 None admin] PATH_INFO: `/v3/groups` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.912941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9dce7-a86e-4b10-a66a-ff449c72c3a7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:35.914364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9dce7-a86e-4b10-a66a-ff449c72c3a7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:35.945953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239bfd0-1adb-4a1c-bec9-64f16f3052a5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:35.983706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075d924-1190-45bf-bf07-724601393329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:35.984350 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075d924-1190-45bf-bf07-724601393329 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:35.984635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075d924-1190-45bf-bf07-724601393329 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:35.984906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075d924-1190-45bf-bf07-724601393329 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:35.994717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075d924-1190-45bf-bf07-724601393329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d075d924-1190-45bf-bf07-724601393329 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:35.999882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:36.000844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 122/247] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:36.002243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075d924-1190-45bf-bf07-724601393329 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:36.006823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 126/248] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Oct 24 13:07:35 2022] POST /identity/v3/groups => generated 226 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:36.007617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9dce7-a86e-4b10-a66a-ff449c72c3a7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:36.071418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:36.124886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9dce7-a86e-4b10-a66a-ff449c72c3a7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:36.131710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075d924-1190-45bf-bf07-724601393329 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:36.151367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:36.162363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:36.162903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:36.163179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:36.163443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:36.164113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:36.164442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:36.166453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:36.171039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda622c6-0dd9-4884-abb1-961a420ed619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:36.171685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda622c6-0dd9-4884-abb1-961a420ed619 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:36.171960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda622c6-0dd9-4884-abb1-961a420ed619 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:36.172319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda622c6-0dd9-4884-abb1-961a420ed619 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:36.175940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 127/249] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:36.182206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:36.184571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda622c6-0dd9-4884-abb1-961a420ed619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-fda622c6-0dd9-4884-abb1-961a420ed619 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:36.190413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:36.191294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:36.192685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 123/250] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:36.197912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda622c6-0dd9-4884-abb1-961a420ed619 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:36.303537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004a47d-0d39-42a3-92ee-51d61b7318b3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:36.313653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda622c6-0dd9-4884-abb1-961a420ed619 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:36.334897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:36.352834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22023aec-f56d-4d0e-849b-7cd24e10fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:36.353502 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22023aec-f56d-4d0e-849b-7cd24e10fc64 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:36.354194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22023aec-f56d-4d0e-849b-7cd24e10fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:36.354194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22023aec-f56d-4d0e-849b-7cd24e10fc64 None admin] PATH_INFO: `/v3/users/4529e93db3b547c98140ff9a504fb311` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:36.355287 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22023aec-f56d-4d0e-849b-7cd24e10fc64 None admin] RBAC: Authorizing `identity:get_user(user_id=4529e93db3b547c98140ff9a504fb311)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:36.356778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22023aec-f56d-4d0e-849b-7cd24e10fc64 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:36.358487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 128/251] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:36 2022] GET /identity/v3/users/4529e93db3b547c98140ff9a504fb311 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:36.358883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:36.359676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:36.360102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:36.360559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:36.361372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:36.361821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:36.363521 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:36.364181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22023aec-f56d-4d0e-849b-7cd24e10fc64 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:36.376526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 124/252] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:36.384909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:36.439549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:36.503741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22023aec-f56d-4d0e-849b-7cd24e10fc64 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:36.517428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfcb42-72d8-43d6-a13f-922e23b7f2d6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:36.560934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702b94f-c937-48dd-912d-b49714ff0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:36.561885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702b94f-c937-48dd-912d-b49714ff0414 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:36.562247 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702b94f-c937-48dd-912d-b49714ff0414 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:36.562625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702b94f-c937-48dd-912d-b49714ff0414 None admin] PATH_INFO: `/v3/system/users/4529e93db3b547c98140ff9a504fb311/roles/ab3629eff4074dd29639a73114e696d4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:36.564483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702b94f-c937-48dd-912d-b49714ff0414 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4529e93db3b547c98140ff9a504fb311, role_id=ab3629eff4074dd29639a73114e696d4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:36.565183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64ef4-3772-4931-934a-9094c0a70319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:36.565821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64ef4-3772-4931-934a-9094c0a70319 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:36.566009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702b94f-c937-48dd-912d-b49714ff0414 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:36.566116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64ef4-3772-4931-934a-9094c0a70319 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:36.566355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f64ef4-3772-4931-934a-9094c0a70319 None admin] PATH_INFO: `/v3/users/81b093e60c0446faa63a11e7c8444f25` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:36.567499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f64ef4-3772-4931-934a-9094c0a70319 None admin] RBAC: Authorizing `identity:get_user(user_id=81b093e60c0446faa63a11e7c8444f25)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:36.568981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f64ef4-3772-4931-934a-9094c0a70319 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:36.570873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 125/253] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:36 2022] GET /identity/v3/users/81b093e60c0446faa63a11e7c8444f25 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:36.578292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64ef4-3772-4931-934a-9094c0a70319 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:36.633126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:36.635511 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 129/254] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Oct 24 13:07:36 2022] PUT /identity/v3/system/users/4529e93db3b547c98140ff9a504fb311/roles/ab3629eff4074dd29639a73114e696d4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:07:36.693729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f64ef4-3772-4931-934a-9094c0a70319 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:36.732797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c03067-c9e2-4f86-890d-4f43ef6a4ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:36.733472 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c03067-c9e2-4f86-890d-4f43ef6a4ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:36.733759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c03067-c9e2-4f86-890d-4f43ef6a4ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:36.734033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c03067-c9e2-4f86-890d-4f43ef6a4ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:36.734780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c03067-c9e2-4f86-890d-4f43ef6a4ae7 None admin] RBAC: Adding query filter params (role.id=eb33c965778e405b95ebf794f31e329a, user.id=81b093e60c0446faa63a11e7c8444f25, scope.system=all) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:36.735097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c03067-c9e2-4f86-890d-4f43ef6a4ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:36.736656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c03067-c9e2-4f86-890d-4f43ef6a4ae7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:36.750740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 126/255] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Oct 24 13:07:36 2022] GET /identity/v3/role_assignments?user.id=81b093e60c0446faa63a11e7c8444f25&scope.system=all&role.id=eb33c965778e405b95ebf794f31e329a => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:38.970162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a154ea-252a-46bb-932a-0f00fa6a197c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:38.970677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a154ea-252a-46bb-932a-0f00fa6a197c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:38.970784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a154ea-252a-46bb-932a-0f00fa6a197c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:38.972401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 130/256] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:38.977412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dddad-4809-49ae-9bf9-2bb40f1f5cb9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:38.977926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dddad-4809-49ae-9bf9-2bb40f1f5cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:38.977999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386dddad-4809-49ae-9bf9-2bb40f1f5cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.000660 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-386dddad-4809-49ae-9bf9-2bb40f1f5cb9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:39.001889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-386dddad-4809-49ae-9bf9-2bb40f1f5cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:39.070321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 127/257] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:07:39.131019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ebaaf-06d0-4585-a005-0ed6883fcfa8 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.131340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ebaaf-06d0-4585-a005-0ed6883fcfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.131589 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ebaaf-06d0-4585-a005-0ed6883fcfa8 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.133238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 131/258] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:39.139211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df95644-45e7-4b46-a0d5-da4b179c67ef None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.139506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df95644-45e7-4b46-a0d5-da4b179c67ef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.139788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df95644-45e7-4b46-a0d5-da4b179c67ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.164853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9df95644-45e7-4b46-a0d5-da4b179c67ef None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:39.166607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9df95644-45e7-4b46-a0d5-da4b179c67ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:39.174525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 128/259] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:07:39.226382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda683e-26e5-4089-9a3b-79e9ef17214f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.226656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda683e-26e5-4089-9a3b-79e9ef17214f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.226850 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda683e-26e5-4089-9a3b-79e9ef17214f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.229046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 132/260] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:39.234517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72006f2d-39eb-4303-b1db-2f97d39b901b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.234807 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72006f2d-39eb-4303-b1db-2f97d39b901b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.235085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72006f2d-39eb-4303-b1db-2f97d39b901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.260540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-72006f2d-39eb-4303-b1db-2f97d39b901b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:39.261741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-72006f2d-39eb-4303-b1db-2f97d39b901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:39.269872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 129/261] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:07:39.486383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda683e-26e5-4089-9a3b-79e9ef17214f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.500934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda683e-26e5-4089-9a3b-79e9ef17214f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.505244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683672c-bc4c-4f27-be59-e37fd0d7d4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:39.505877 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683672c-bc4c-4f27-be59-e37fd0d7d4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.506155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683672c-bc4c-4f27-be59-e37fd0d7d4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.506429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683672c-bc4c-4f27-be59-e37fd0d7d4d1 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.525973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683672c-bc4c-4f27-be59-e37fd0d7d4d1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-3683672c-bc4c-4f27-be59-e37fd0d7d4d1 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:39.531494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:39.533714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:39.535626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 133/262] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:39.540313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72006f2d-39eb-4303-b1db-2f97d39b901b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.553258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72006f2d-39eb-4303-b1db-2f97d39b901b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.561292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:39.561907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.562186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.562460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.563128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:39.563436 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:39.567410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:07:39.567410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Oct 24 13:07:39.567410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: Oct 24 13:07:39.568367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:39.584385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 130/263] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:39.590560 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683672c-bc4c-4f27-be59-e37fd0d7d4d1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.600913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683672c-bc4c-4f27-be59-e37fd0d7d4d1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.608315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:39.608946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.609225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.609496 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] PATH_INFO: `/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.616049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] RBAC: Authorizing `identity:get_group(group_id=a750fcc292524a33b8ffb95f32a2ba6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:39.617911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:07:39.617911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:07:39.617911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:07:39.619181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:39.621108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 134/264] 10.222.0.48 () {56 vars in 1201 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:39.623677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.632465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.638964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d4437-dd99-446f-a7d7-ab39cda50cf6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.647632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b906-ce61-427c-9989-075bfa2019a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:39.648259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b906-ce61-427c-9989-075bfa2019a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.648562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b906-ce61-427c-9989-075bfa2019a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.648838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b906-ce61-427c-9989-075bfa2019a0 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.652214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f409d-934e-47a7-b562-d62e5f9dc060 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.664756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b7da4-2bd6-45c3-a925-453c7db12749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:39.665461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b7da4-2bd6-45c3-a925-453c7db12749 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.666222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b7da4-2bd6-45c3-a925-453c7db12749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.666495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b7da4-2bd6-45c3-a925-453c7db12749 admin admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.667745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b906-ce61-427c-9989-075bfa2019a0 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e626b906-ce61-427c-9989-075bfa2019a0 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:39.672971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:39.675134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:39.677105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 131/265] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:39.683627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b7da4-2bd6-45c3-a925-453c7db12749 admin admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:39.683627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b7da4-2bd6-45c3-a925-453c7db12749 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:39.683627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 135/266] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:39.683627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b7da4-2bd6-45c3-a925-453c7db12749 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.684347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b259ff-7822-458c-a69a-14c2a71f46fa None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.684347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b259ff-7822-458c-a69a-14c2a71f46fa None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.684347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b259ff-7822-458c-a69a-14c2a71f46fa None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.684347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 132/267] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:39.692781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9240a0-fd47-4088-8fb0-f378566f17a9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.693070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9240a0-fd47-4088-8fb0-f378566f17a9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.693342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9240a0-fd47-4088-8fb0-f378566f17a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.702653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b7da4-2bd6-45c3-a925-453c7db12749 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.716868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:39.717471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.717753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.718026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.721947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:39.722257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:39.723267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:07:39.723267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:07:39.723267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:07:39.724144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:39.726357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-da9240a0-fd47-4088-8fb0-f378566f17a9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:39.727550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-da9240a0-fd47-4088-8fb0-f378566f17a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:39.743465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 136/268] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:39.743967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.757569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:39.762867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2a37-bd8e-41d4-8364-b13e585e751c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.767318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:39.768039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.768238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.768576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c/groups/a750fcc292524a33b8ffb95f32a2ba6c/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.770460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add9f115b9a84620af8483f8863c3f6c, group_id=a750fcc292524a33b8ffb95f32a2ba6c, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:39.771479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:07:39.771479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:07:39.771479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:07:39.773473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:39.819200 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 133/269] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:39.820176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9240a0-fd47-4088-8fb0-f378566f17a9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.834551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9240a0-fd47-4088-8fb0-f378566f17a9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.845560 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 137/270] 10.222.0.48 () {58 vars in 1463 bytes} [Mon Oct 24 13:07:39 2022] PUT /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c/groups/a750fcc292524a33b8ffb95f32a2ba6c/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:39.846445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.853687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3326-59e7-40c0-bd13-62d76c392a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:39.855627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3326-59e7-40c0-bd13-62d76c392a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:39.855905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3326-59e7-40c0-bd13-62d76c392a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:39.856176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab3326-59e7-40c0-bd13-62d76c392a17 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:39.882325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab3326-59e7-40c0-bd13-62d76c392a17 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2cab3326-59e7-40c0-bd13-62d76c392a17 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:39.895130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:39.897368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:39.899355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 134/271] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 77 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:39.904303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3326-59e7-40c0-bd13-62d76c392a17 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:39.921141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab3326-59e7-40c0-bd13-62d76c392a17 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:39.994323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f2846-d6b0-4c1f-9077-5c7bc1686c55 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.002162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c7d1-6b3b-4573-b110-996600d4744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.003060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c7d1-6b3b-4573-b110-996600d4744d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.003337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c7d1-6b3b-4573-b110-996600d4744d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.003608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c7d1-6b3b-4573-b110-996600d4744d admin admin] PATH_INFO: `/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.004767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6c7d1-6b3b-4573-b110-996600d4744d admin admin] RBAC: Authorizing `identity:get_group(group_id=a750fcc292524a33b8ffb95f32a2ba6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.006726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6c7d1-6b3b-4573-b110-996600d4744d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.010045 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 138/272] 10.222.0.48 () {56 vars in 1201 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:40.018024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c7d1-6b3b-4573-b110-996600d4744d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.031905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c7d1-6b3b-4573-b110-996600d4744d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.042678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c0116-ffae-4b84-8d06-aabe456d0586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.043334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0116-ffae-4b84-8d06-aabe456d0586 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.043617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0116-ffae-4b84-8d06-aabe456d0586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.043891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c0116-ffae-4b84-8d06-aabe456d0586 admin admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.045217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c0116-ffae-4b84-8d06-aabe456d0586 admin admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.047649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c0116-ffae-4b84-8d06-aabe456d0586 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.050202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 139/273] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Oct 24 13:07:40 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:40.055977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c0116-ffae-4b84-8d06-aabe456d0586 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.070938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.072442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.072728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.073001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.073673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:40.073980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.074239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c0116-ffae-4b84-8d06-aabe456d0586 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.075604 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.092308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8d01f-d5e4-456c-8d46-ec21b0b9bd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.092308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8d01f-d5e4-456c-8d46-ec21b0b9bd37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.092308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8d01f-d5e4-456c-8d46-ec21b0b9bd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.092308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8d01f-d5e4-456c-8d46-ec21b0b9bd37 admin admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d/groups/a750fcc292524a33b8ffb95f32a2ba6c/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.093417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8d01f-d5e4-456c-8d46-ec21b0b9bd37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312738b3aa6f46e2bab330e5c745742d, group_id=a750fcc292524a33b8ffb95f32a2ba6c, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.093417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8d01f-d5e4-456c-8d46-ec21b0b9bd37 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.093686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 135/274] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Oct 24 13:07:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:40.099478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.116990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51644f-cd5d-4cc0-bfa0-fc714a89f4dc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.124722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.125384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.125652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.125925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] PATH_INFO: `/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.127362 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] RBAC: Authorizing `identity:get_group(group_id=a750fcc292524a33b8ffb95f32a2ba6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.128413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:07:40.128413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Oct 24 13:07:40.128413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: Oct 24 13:07:40.129875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.131791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 136/275] 10.222.0.48 () {56 vars in 1201 bytes} [Mon Oct 24 13:07:40 2022] GET /identity/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c => generated 226 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:40.138126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.152992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c924e-21d3-4685-be17-5dc5f0fcec0e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.161372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd15a77-f415-45a5-92c1-4684689bde73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.162055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd15a77-f415-45a5-92c1-4684689bde73 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.162343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd15a77-f415-45a5-92c1-4684689bde73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.162619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd15a77-f415-45a5-92c1-4684689bde73 admin admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.165292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd15a77-f415-45a5-92c1-4684689bde73 admin admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.167939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd15a77-f415-45a5-92c1-4684689bde73 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.170575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 140/277] 10.222.0.48 () {58 vars in 1463 bytes} [Mon Oct 24 13:07:40 2022] PUT /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d/groups/a750fcc292524a33b8ffb95f32a2ba6c/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:07:40.170809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 137/277] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Oct 24 13:07:40 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:40.176068 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8d01f-d5e4-456c-8d46-ec21b0b9bd37 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.177493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd15a77-f415-45a5-92c1-4684689bde73 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.181660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:40.198055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd15a77-f415-45a5-92c1-4684689bde73 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.284451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.285584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.286109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.286592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c/groups/a750fcc292524a33b8ffb95f32a2ba6c/roles/23a3d7681bf247468a0baa9bfed6f4e9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.290116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add9f115b9a84620af8483f8863c3f6c, group_id=a750fcc292524a33b8ffb95f32a2ba6c, role_id=23a3d7681bf247468a0baa9bfed6f4e9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.291522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:07:40.291522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Oct 24 13:07:40.291522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: Oct 24 13:07:40.294367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.373029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8d01f-d5e4-456c-8d46-ec21b0b9bd37 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.374871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 138/278] 10.222.0.48 () {58 vars in 1463 bytes} [Mon Oct 24 13:07:40 2022] PUT /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c/groups/a750fcc292524a33b8ffb95f32a2ba6c/roles/23a3d7681bf247468a0baa9bfed6f4e9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:40.428938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c6f87-1797-4a42-8fb2-c61ba9a51e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.429674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c6f87-1797-4a42-8fb2-c61ba9a51e97 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.429959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c6f87-1797-4a42-8fb2-c61ba9a51e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.430231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c6f87-1797-4a42-8fb2-c61ba9a51e97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.447198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c6f87-1797-4a42-8fb2-c61ba9a51e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ee9c6f87-1797-4a42-8fb2-c61ba9a51e97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:40.455040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:40.456013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:40.456926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 141/279] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:40.464514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.484293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:40.592912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde89398-7619-4135-8096-ec59acb21068 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.632214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.632863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.633143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.633414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.634642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:40.634956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.636359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.645565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 139/280] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:40.651307 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c6f87-1797-4a42-8fb2-c61ba9a51e97 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.743062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:40.785766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c6f87-1797-4a42-8fb2-c61ba9a51e97 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:40.829780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb87f7-ab42-4939-b6dc-aa288749d7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:40.830439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb87f7-ab42-4939-b6dc-aa288749d7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:40.830721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb87f7-ab42-4939-b6dc-aa288749d7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:40.830994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb87f7-ab42-4939-b6dc-aa288749d7f8 None admin] PATH_INFO: `/v3/users/4529e93db3b547c98140ff9a504fb311` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:40.832194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb87f7-ab42-4939-b6dc-aa288749d7f8 None admin] RBAC: Authorizing `identity:get_user(user_id=4529e93db3b547c98140ff9a504fb311)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:40.833821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb87f7-ab42-4939-b6dc-aa288749d7f8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:40.835753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 142/281] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:07:40 2022] GET /identity/v3/users/4529e93db3b547c98140ff9a504fb311 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:40.841997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:40.945385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:40.975221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad58797-504e-45f3-bbd4-008ee44106be None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.018089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df17994-68eb-4e4c-a1e6-5e9c6605ce19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.018692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17994-68eb-4e4c-a1e6-5e9c6605ce19 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.018968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17994-68eb-4e4c-a1e6-5e9c6605ce19 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.019239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17994-68eb-4e4c-a1e6-5e9c6605ce19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.019961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df17994-68eb-4e4c-a1e6-5e9c6605ce19 None admin] RBAC: Adding query filter params (role.id=ab3629eff4074dd29639a73114e696d4, user.id=4529e93db3b547c98140ff9a504fb311, scope.system=all) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:41.020260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df17994-68eb-4e4c-a1e6-5e9c6605ce19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:41.023211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aaa86-7b4c-46df-911d-2cc339593725 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.023650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aaa86-7b4c-46df-911d-2cc339593725 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.023815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aaa86-7b4c-46df-911d-2cc339593725 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.025280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df17994-68eb-4e4c-a1e6-5e9c6605ce19 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:41.025972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 143/282] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:41.031350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058b19-6915-41f8-b478-a4c7c7e80ab3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.031747 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058b19-6915-41f8-b478-a4c7c7e80ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.032021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058b19-6915-41f8-b478-a4c7c7e80ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.038913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 140/283] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Oct 24 13:07:40 2022] GET /identity/v3/role_assignments?user.id=4529e93db3b547c98140ff9a504fb311&scope.system=all&role.id=ab3629eff4074dd29639a73114e696d4 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:41.055737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b3058b19-6915-41f8-b478-a4c7c7e80ab3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:41.057039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b3058b19-6915-41f8-b478-a4c7c7e80ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:41.135070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 144/284] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 13:07:41.371769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a32fb1-bfe1-4626-b2b6-3732a62cd098 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.372825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a32fb1-bfe1-4626-b2b6-3732a62cd098 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.373303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a32fb1-bfe1-4626-b2b6-3732a62cd098 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.375115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 141/285] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:41.380151 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c25eef-3d33-468d-a0bc-cf08baccd8bb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.380475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c25eef-3d33-468d-a0bc-cf08baccd8bb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.380753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c25eef-3d33-468d-a0bc-cf08baccd8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.406001 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b3c25eef-3d33-468d-a0bc-cf08baccd8bb None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:41.407505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b3c25eef-3d33-468d-a0bc-cf08baccd8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:41.417834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 145/286] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 13:07:41.625511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a32fb1-bfe1-4626-b2b6-3732a62cd098 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.642202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a32fb1-bfe1-4626-b2b6-3732a62cd098 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.646949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca7e1-2f48-425e-8673-06badbd8df4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.647785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca7e1-2f48-425e-8673-06badbd8df4b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.648283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca7e1-2f48-425e-8673-06badbd8df4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.648751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca7e1-2f48-425e-8673-06badbd8df4b admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.662669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ca7e1-2f48-425e-8673-06badbd8df4b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-113ca7e1-2f48-425e-8673-06badbd8df4b admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:41.667774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:41.668694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:41.669509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:41.671733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 142/287] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Oct 24 13:07:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:41.677594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c25eef-3d33-468d-a0bc-cf08baccd8bb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.687724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c25eef-3d33-468d-a0bc-cf08baccd8bb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.692499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.693316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.693786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.694264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.695173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:41.695665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:41.698064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:41.711962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 146/288] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Oct 24 13:07:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:41.718748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca7e1-2f48-425e-8673-06badbd8df4b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.725001 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.734862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca7e1-2f48-425e-8673-06badbd8df4b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.744894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95d1a0-5f9c-4651-a467-b83624446ce0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.745712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33a0b-fb95-4121-b30e-07872e25260f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.746551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33a0b-fb95-4121-b30e-07872e25260f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.747042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33a0b-fb95-4121-b30e-07872e25260f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.747501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33a0b-fb95-4121-b30e-07872e25260f admin admin] PATH_INFO: `/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.749459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e33a0b-fb95-4121-b30e-07872e25260f admin admin] RBAC: Authorizing `identity:get_group(group_id=a750fcc292524a33b8ffb95f32a2ba6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:41.751547 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e33a0b-fb95-4121-b30e-07872e25260f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:41.753361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 143/289] 10.222.0.48 () {56 vars in 1201 bytes} [Mon Oct 24 13:07:41 2022] GET /identity/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c => generated 226 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:41.757559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49bd42-cb6f-4366-9841-1971f897880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.758530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49bd42-cb6f-4366-9841-1971f897880a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.760101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33a0b-fb95-4121-b30e-07872e25260f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.760954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49bd42-cb6f-4366-9841-1971f897880a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.761767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49bd42-cb6f-4366-9841-1971f897880a admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.777426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33a0b-fb95-4121-b30e-07872e25260f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.788303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49bd42-cb6f-4366-9841-1971f897880a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-cb49bd42-cb6f-4366-9841-1971f897880a admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:41.794604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:41.795528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:41.796383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:41.798090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9f997-a34f-4160-a53b-fee387d16247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.798939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9f997-a34f-4160-a53b-fee387d16247 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.799403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9f997-a34f-4160-a53b-fee387d16247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.799905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9f997-a34f-4160-a53b-fee387d16247 admin admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.801300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9f997-a34f-4160-a53b-fee387d16247 admin admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:41.803536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9f997-a34f-4160-a53b-fee387d16247 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:41.805397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 144/290] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Oct 24 13:07:41 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:41.807650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 147/291] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Oct 24 13:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 87 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:41.811784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9f997-a34f-4160-a53b-fee387d16247 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.813237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49bd42-cb6f-4366-9841-1971f897880a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.832984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9f997-a34f-4160-a53b-fee387d16247 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.834756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49bd42-cb6f-4366-9841-1971f897880a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.842201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa070b-1668-4760-88df-7ae200a71ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.843825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa070b-1668-4760-88df-7ae200a71ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.844571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.845415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.845884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa070b-1668-4760-88df-7ae200a71ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.846347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.846806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa070b-1668-4760-88df-7ae200a71ba2 admin admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d/groups/a750fcc292524a33b8ffb95f32a2ba6c/roles/23a3d7681bf247468a0baa9bfed6f4e9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.847263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.850483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:41.851388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:41.853971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:41.854849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa070b-1668-4760-88df-7ae200a71ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312738b3aa6f46e2bab330e5c745742d, group_id=a750fcc292524a33b8ffb95f32a2ba6c, role_id=23a3d7681bf247468a0baa9bfed6f4e9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:41.857677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa070b-1668-4760-88df-7ae200a71ba2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:41.870189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 148/292] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Oct 24 13:07:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:41.882821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.888412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39556e3a-a13b-4442-bb6f-6ed594b72d4c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.893194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad3dc-b665-46d9-a3c7-673a3de88fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.894120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad3dc-b665-46d9-a3c7-673a3de88fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.894592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad3dc-b665-46d9-a3c7-673a3de88fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.895101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ad3dc-b665-46d9-a3c7-673a3de88fc9 admin admin] PATH_INFO: `/v3/groups/d5bec7ecc51a45a3bbd92531325495bf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.901109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad3dc-b665-46d9-a3c7-673a3de88fc9 admin admin] RBAC: Authorizing `identity:get_group(group_id=d5bec7ecc51a45a3bbd92531325495bf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:41.905108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ad3dc-b665-46d9-a3c7-673a3de88fc9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:41.907400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 149/293] 10.222.0.48 () {56 vars in 1201 bytes} [Mon Oct 24 13:07:41 2022] GET /identity/v3/groups/d5bec7ecc51a45a3bbd92531325495bf => generated 229 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:41.912394 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad3dc-b665-46d9-a3c7-673a3de88fc9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.923352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ad3dc-b665-46d9-a3c7-673a3de88fc9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:41.928424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a31bf9-1f92-49f6-83e3-f7530ae80ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:41.929318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a31bf9-1f92-49f6-83e3-f7530ae80ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:41.929840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a31bf9-1f92-49f6-83e3-f7530ae80ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:41.930370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a31bf9-1f92-49f6-83e3-f7530ae80ba4 admin admin] PATH_INFO: `/v3/projects/c5f7bf50f5a94c7694720e6be18c9de2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:41.932412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a31bf9-1f92-49f6-83e3-f7530ae80ba4 admin admin] RBAC: Authorizing `identity:get_project(project_id=c5f7bf50f5a94c7694720e6be18c9de2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:41.934546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a31bf9-1f92-49f6-83e3-f7530ae80ba4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:41.936847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 150/294] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Oct 24 13:07:41 2022] GET /identity/v3/projects/c5f7bf50f5a94c7694720e6be18c9de2 => generated 344 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:41.942234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a31bf9-1f92-49f6-83e3-f7530ae80ba4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:41.954291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 145/295] 10.222.0.48 () {58 vars in 1463 bytes} [Mon Oct 24 13:07:41 2022] PUT /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d/groups/a750fcc292524a33b8ffb95f32a2ba6c/roles/23a3d7681bf247468a0baa9bfed6f4e9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:07:41.961279 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a31bf9-1f92-49f6-83e3-f7530ae80ba4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:42.001855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:42.037212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d07ce-6d3b-4d28-b49f-6d9830379ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:42.037996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d07ce-6d3b-4d28-b49f-6d9830379ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:42.038182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d07ce-6d3b-4d28-b49f-6d9830379ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:42.038401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d07ce-6d3b-4d28-b49f-6d9830379ec9 admin admin] PATH_INFO: `/v3/projects/c5f7bf50f5a94c7694720e6be18c9de2/groups/d5bec7ecc51a45a3bbd92531325495bf/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:42.040137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d07ce-6d3b-4d28-b49f-6d9830379ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f7bf50f5a94c7694720e6be18c9de2, group_id=d5bec7ecc51a45a3bbd92531325495bf, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:42.042939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d07ce-6d3b-4d28-b49f-6d9830379ec9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:42.132063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 151/296] 10.222.0.48 () {58 vars in 1463 bytes} [Mon Oct 24 13:07:41 2022] PUT /identity/v3/projects/c5f7bf50f5a94c7694720e6be18c9de2/groups/d5bec7ecc51a45a3bbd92531325495bf/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:43.233712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c72b-5771-4b33-9de1-b6f178fad530 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:43.234219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c72b-5771-4b33-9de1-b6f178fad530 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:43.234219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c72b-5771-4b33-9de1-b6f178fad530 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:43.235554 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 146/297] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:43.241084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77374-12de-4702-9c4f-7be095717cbf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:43.241517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77374-12de-4702-9c4f-7be095717cbf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:43.241693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77374-12de-4702-9c4f-7be095717cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:43.248379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86863-4847-4335-81da-27140a661743 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:43.248663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86863-4847-4335-81da-27140a661743 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:43.248936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86863-4847-4335-81da-27140a661743 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:43.250242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 147/298] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:07:43.255440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1c50-3ccf-4637-b854-ea63cc525c1a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:43.255720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1c50-3ccf-4637-b854-ea63cc525c1a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:43.255992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1c50-3ccf-4637-b854-ea63cc525c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:43.282920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-91e77374-12de-4702-9c4f-7be095717cbf None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:43.284109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-91e77374-12de-4702-9c4f-7be095717cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:43.290935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a52f1c50-3ccf-4637-b854-ea63cc525c1a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:43.292089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a52f1c50-3ccf-4637-b854-ea63cc525c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:43.407182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 148/299] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:43.409408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 152/300] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:43.631112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134d70-008e-481a-ad99-9fbb9b353920 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:43.631613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134d70-008e-481a-ad99-9fbb9b353920 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:43.632017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134d70-008e-481a-ad99-9fbb9b353920 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:43.633732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 149/301] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:43.642739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b756c1-d76c-4ca7-937b-3029199b77f8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:43.643093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b756c1-d76c-4ca7-937b-3029199b77f8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:43.643432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b756c1-d76c-4ca7-937b-3029199b77f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:43.662848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b9b756c1-d76c-4ca7-937b-3029199b77f8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:43.663938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b9b756c1-d76c-4ca7-937b-3029199b77f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:43.721779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63134d70-008e-481a-ad99-9fbb9b353920 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:43.751634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:43.752903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 153/302] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:43.767083 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b756c1-d76c-4ca7-937b-3029199b77f8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:43.815181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:43.877504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63134d70-008e-481a-ad99-9fbb9b353920 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:43.932220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b756c1-d76c-4ca7-937b-3029199b77f8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:43.941893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90f8c4-04a7-400e-a36c-7663f6f9b47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:43.942533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f8c4-04a7-400e-a36c-7663f6f9b47d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:43.942808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f8c4-04a7-400e-a36c-7663f6f9b47d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:43.943078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90f8c4-04a7-400e-a36c-7663f6f9b47d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:43.956456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90f8c4-04a7-400e-a36c-7663f6f9b47d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0f90f8c4-04a7-400e-a36c-7663f6f9b47d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:43.966377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:43.967289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:43.968377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 150/303] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:43.974288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90f8c4-04a7-400e-a36c-7663f6f9b47d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:43.989092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102aac7-79af-4c1c-ae5b-de1492b8b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:43.989737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102aac7-79af-4c1c-ae5b-de1492b8b145 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:43.990069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102aac7-79af-4c1c-ae5b-de1492b8b145 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:43.990344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102aac7-79af-4c1c-ae5b-de1492b8b145 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.015991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102aac7-79af-4c1c-ae5b-de1492b8b145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d102aac7-79af-4c1c-ae5b-de1492b8b145 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:44.026389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:44.027298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:44.029666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 154/304] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:44.033548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102aac7-79af-4c1c-ae5b-de1492b8b145 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:44.123125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:44.137808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90f8c4-04a7-400e-a36c-7663f6f9b47d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:44.177443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:44.178044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.178320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.178590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.179246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:44.179543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:44.181488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:44.182618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:44.191000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 151/305] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:44.191904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:44.197112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102aac7-79af-4c1c-ae5b-de1492b8b145 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:44.235593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:44.236537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.236985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.237393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.238244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] RBAC: Adding query filter params (name=member) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:44.238683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:44.241050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:44.252769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 155/306] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:44.254212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:44.318695 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b548-f71c-4fa0-b534-151967d0c999 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:44.361496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3d000-6e18-4110-9f17-ed7d98860127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:44.362859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3d000-6e18-4110-9f17-ed7d98860127 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.363423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3d000-6e18-4110-9f17-ed7d98860127 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.363997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3d000-6e18-4110-9f17-ed7d98860127 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.366839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ac5eb-c6d6-4450-a305-af05e722453e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:44.377786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3d000-6e18-4110-9f17-ed7d98860127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9ab3d000-6e18-4110-9f17-ed7d98860127 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:44.386668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:44.387585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:44.388439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:44.389867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 152/307] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:44.394158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3d000-6e18-4110-9f17-ed7d98860127 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:44.418694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e3d99-779b-4617-8988-16cb06784de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:44.419974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3d99-779b-4617-8988-16cb06784de2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.420474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3d99-779b-4617-8988-16cb06784de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.420956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3d99-779b-4617-8988-16cb06784de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.450849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:44.451027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:44.510486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3d000-6e18-4110-9f17-ed7d98860127 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:44.510979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e3d99-779b-4617-8988-16cb06784de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:44.514552 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e3d99-779b-4617-8988-16cb06784de2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:44.547642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091aaea-1ac8-4d9a-bed1-6b66ce4005a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:44.548567 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091aaea-1ac8-4d9a-bed1-6b66ce4005a1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.549059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091aaea-1ac8-4d9a-bed1-6b66ce4005a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.549506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091aaea-1ac8-4d9a-bed1-6b66ce4005a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.631157 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091aaea-1ac8-4d9a-bed1-6b66ce4005a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:44.632039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 156/308] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:44.633020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e3d99-779b-4617-8988-16cb06784de2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:44.633457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091aaea-1ac8-4d9a-bed1-6b66ce4005a1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:44.646814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:44.690512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:44.747309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 153/309] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:44.747544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e3d99-779b-4617-8988-16cb06784de2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:44.748656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091aaea-1ac8-4d9a-bed1-6b66ce4005a1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:44.785094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:44.785701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.785977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.786266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.787172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:44.787172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:44.789151 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:44.797567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 157/310] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:44.798393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:44.860311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091aaea-1ac8-4d9a-bed1-6b66ce4005a1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:44.872574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:44.909555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e4e00-dd9f-402b-8280-89157860988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:44.910198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e4e00-dd9f-402b-8280-89157860988a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.910482 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e4e00-dd9f-402b-8280-89157860988a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.911029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e4e00-dd9f-402b-8280-89157860988a None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.912654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e4e00-dd9f-402b-8280-89157860988a None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:44.914362 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e4e00-dd9f-402b-8280-89157860988a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:44.916713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 154/311] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:44.917650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e4e00-dd9f-402b-8280-89157860988a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:44.923961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6937e66-bdd4-4101-a2f8-36af6cabdbee None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:44.938888 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:44.970184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc5089-6274-4216-a5c7-4512fa10cd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:44.970619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc5089-6274-4216-a5c7-4512fa10cd48 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.970904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc5089-6274-4216-a5c7-4512fa10cd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.971181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc5089-6274-4216-a5c7-4512fa10cd48 None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.976104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc5089-6274-4216-a5c7-4512fa10cd48 None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:44.977641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc5089-6274-4216-a5c7-4512fa10cd48 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:44.980045 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 158/312] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:44.982239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f371c-c527-4c39-98da-1622a1828200 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:44.982239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f371c-c527-4c39-98da-1622a1828200 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:44.982239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f371c-c527-4c39-98da-1622a1828200 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:44.987701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 159/313] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:44.989292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f371c-c527-4c39-98da-1622a1828200 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:45.050614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e4e00-dd9f-402b-8280-89157860988a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:45.099738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae71ce-816f-45d9-b5cf-9cbf0ca8a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:45.100377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae71ce-816f-45d9-b5cf-9cbf0ca8a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.100657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae71ce-816f-45d9-b5cf-9cbf0ca8a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.100928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae71ce-816f-45d9-b5cf-9cbf0ca8a3b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.129735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f371c-c527-4c39-98da-1622a1828200 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:45.159002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:45.166739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd22611-5976-4106-a23c-93432ecace01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:45.171971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22611-5976-4106-a23c-93432ecace01 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.171971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22611-5976-4106-a23c-93432ecace01 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.171971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22611-5976-4106-a23c-93432ecace01 None admin] PATH_INFO: `/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.171971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd22611-5976-4106-a23c-93432ecace01 None admin] RBAC: Authorizing `identity:get_group(group_id=a750fcc292524a33b8ffb95f32a2ba6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:45.171971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd22611-5976-4106-a23c-93432ecace01 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:45.172349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 160/314] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:45.176533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd22611-5976-4106-a23c-93432ecace01 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:45.183169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae71ce-816f-45d9-b5cf-9cbf0ca8a3b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:45.184830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae71ce-816f-45d9-b5cf-9cbf0ca8a3b8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:45.236151 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:45.299963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd22611-5976-4106-a23c-93432ecace01 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:45.301132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 155/315] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:07:45.304190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb012-d418-4df1-9376-67b6f30b2138 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.304190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb012-d418-4df1-9376-67b6f30b2138 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.304190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb012-d418-4df1-9376-67b6f30b2138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.328492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e0ecb012-d418-4df1-9376-67b6f30b2138 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:45.329700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e0ecb012-d418-4df1-9376-67b6f30b2138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:45.345718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6424e0f-a2b3-4ec7-9af5-107258cd19e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:45.346453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424e0f-a2b3-4ec7-9af5-107258cd19e3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.346768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424e0f-a2b3-4ec7-9af5-107258cd19e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.347043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424e0f-a2b3-4ec7-9af5-107258cd19e3 None admin] PATH_INFO: `/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.348201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6424e0f-a2b3-4ec7-9af5-107258cd19e3 None admin] RBAC: Authorizing `identity:get_group(group_id=a750fcc292524a33b8ffb95f32a2ba6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:45.349689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6424e0f-a2b3-4ec7-9af5-107258cd19e3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:45.351269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 161/316] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Oct 24 13:07:45 2022] GET /identity/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:45.352464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ff745-48fe-496d-b491-1bc7750aba0f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.352783 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ff745-48fe-496d-b491-1bc7750aba0f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.353060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ff745-48fe-496d-b491-1bc7750aba0f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.354249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 162/317] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:45.355073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ff745-48fe-496d-b491-1bc7750aba0f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:45.416129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 156/318] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:45.416839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecb012-d418-4df1-9376-67b6f30b2138 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:45.427556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:45.470944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ff745-48fe-496d-b491-1bc7750aba0f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:45.489989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:45.516694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:45.516694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.516979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.517192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.517856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] RBAC: Adding query filter params (group.id=a750fcc292524a33b8ffb95f32a2ba6c, role.id=eb33c965778e405b95ebf794f31e329a, scope.project.id=312738b3aa6f46e2bab330e5c745742d) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:45.518167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:45.519607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:45.527428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 163/319] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Oct 24 13:07:45 2022] GET /identity/v3/role_assignments?group.id=a750fcc292524a33b8ffb95f32a2ba6c&scope.project.id=312738b3aa6f46e2bab330e5c745742d&role.id=eb33c965778e405b95ebf794f31e329a => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:45.527569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:45.544224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecb012-d418-4df1-9376-67b6f30b2138 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:45.592745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d67ee0-d39d-4161-9909-17c336bae85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:45.593753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67ee0-d39d-4161-9909-17c336bae85c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.594033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67ee0-d39d-4161-9909-17c336bae85c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.594307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67ee0-d39d-4161-9909-17c336bae85c None admin] PATH_INFO: `/v3/projects/add9f115b9a84620af8483f8863c3f6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.595661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d67ee0-d39d-4161-9909-17c336bae85c None admin] RBAC: Authorizing `identity:get_project(project_id=add9f115b9a84620af8483f8863c3f6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:45.597187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d67ee0-d39d-4161-9909-17c336bae85c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:45.600059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 157/320] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:45 2022] GET /identity/v3/projects/add9f115b9a84620af8483f8863c3f6c => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:45.601344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0adca-8b88-4141-bc4e-24b949006a3f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.601625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0adca-8b88-4141-bc4e-24b949006a3f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.601895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0adca-8b88-4141-bc4e-24b949006a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.626587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-36b0adca-8b88-4141-bc4e-24b949006a3f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:45.627979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-36b0adca-8b88-4141-bc4e-24b949006a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:45.674213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83027abb-b7f0-4b2b-a6bf-4bafe4043945 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:45.714089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 158/321] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:45.715668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0adca-8b88-4141-bc4e-24b949006a3f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:45.720480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:45.721090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.721368 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.721641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.722418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] RBAC: Adding query filter params (group.id=a750fcc292524a33b8ffb95f32a2ba6c, role.id=eb33c965778e405b95ebf794f31e329a, scope.project.id=add9f115b9a84620af8483f8863c3f6c) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:45.722730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:45.724532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:45.724729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:45.733163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 164/322] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Oct 24 13:07:45 2022] GET /identity/v3/role_assignments?group.id=a750fcc292524a33b8ffb95f32a2ba6c&scope.project.id=add9f115b9a84620af8483f8863c3f6c&role.id=eb33c965778e405b95ebf794f31e329a => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:45.737268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:45.810323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:45.875591 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0adca-8b88-4141-bc4e-24b949006a3f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:45.893435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58826592-381b-4116-bdb8-455e5c0a2844 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:45.926597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed8c9b-e071-422f-aa6e-7dd060c9b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:45.927394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed8c9b-e071-422f-aa6e-7dd060c9b4da None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.927675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed8c9b-e071-422f-aa6e-7dd060c9b4da None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.928118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed8c9b-e071-422f-aa6e-7dd060c9b4da None admin] PATH_INFO: `/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.929944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed8c9b-e071-422f-aa6e-7dd060c9b4da None admin] RBAC: Authorizing `identity:get_group(group_id=a750fcc292524a33b8ffb95f32a2ba6c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:45.932204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed8c9b-e071-422f-aa6e-7dd060c9b4da None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:45.934173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6c0f6-60ce-4a85-a6cc-1f2fa9263b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:45.934816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6c0f6-60ce-4a85-a6cc-1f2fa9263b88 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:45.934918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 159/323] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Oct 24 13:07:45 2022] GET /identity/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:45.935149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6c0f6-60ce-4a85-a6cc-1f2fa9263b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:45.935425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6c0f6-60ce-4a85-a6cc-1f2fa9263b88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:45.940830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed8c9b-e071-422f-aa6e-7dd060c9b4da None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:45.947796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6c0f6-60ce-4a85-a6cc-1f2fa9263b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-38a6c0f6-60ce-4a85-a6cc-1f2fa9263b88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:45.953518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:45.954463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:45.955384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 165/324] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:45.965714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6c0f6-60ce-4a85-a6cc-1f2fa9263b88 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.058165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:46.087297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed8c9b-e071-422f-aa6e-7dd060c9b4da None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.091628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6c0f6-60ce-4a85-a6cc-1f2fa9263b88 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.095914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:46.132263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.133720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.133720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.133720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.134008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.134137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:46.134266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.134436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:46.134550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.134753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.135476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] RBAC: Adding query filter params (group.id=a750fcc292524a33b8ffb95f32a2ba6c, role.id=23a3d7681bf247468a0baa9bfed6f4e9, scope.project.id=add9f115b9a84620af8483f8863c3f6c) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:46.135781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:46.136758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:46.137219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:46.143947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 160/325] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Oct 24 13:07:45 2022] GET /identity/v3/role_assignments?group.id=a750fcc292524a33b8ffb95f32a2ba6c&scope.project.id=add9f115b9a84620af8483f8863c3f6c&role.id=23a3d7681bf247468a0baa9bfed6f4e9 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:46.144788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.147536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 166/326] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:46.153675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.281551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1f95a-26cf-4de2-bce4-a352689e952e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.283730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653db6b-9a3c-4d00-98f0-62a523c10457 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.319832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa00534-28ee-4323-8366-2a2fc5de7e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.321263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa00534-28ee-4323-8366-2a2fc5de7e51 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.321959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c6923-f213-44d5-a1e2-d2c8fd5345d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.322232 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa00534-28ee-4323-8366-2a2fc5de7e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.322509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa00534-28ee-4323-8366-2a2fc5de7e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.322605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c6923-f213-44d5-a1e2-d2c8fd5345d4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.322797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c6923-f213-44d5-a1e2-d2c8fd5345d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.323114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c6923-f213-44d5-a1e2-d2c8fd5345d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.330887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:46.335649 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c6923-f213-44d5-a1e2-d2c8fd5345d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-902c6923-f213-44d5-a1e2-d2c8fd5345d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:46.340501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:46.342568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:46.343646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 161/327] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:07:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:46.348085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c6923-f213-44d5-a1e2-d2c8fd5345d4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.363310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:46.403690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa00534-28ee-4323-8366-2a2fc5de7e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:46.405216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa00534-28ee-4323-8366-2a2fc5de7e51 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:46.470579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c6923-f213-44d5-a1e2-d2c8fd5345d4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.507414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.508514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.508514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.508682 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.509389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:46.509691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:46.511048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:46.519925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 162/328] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:46.525281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 167/329] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 13:07:46.528051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.530660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa00534-28ee-4323-8366-2a2fc5de7e51 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.595858 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:46.604306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:46.639927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa00534-28ee-4323-8366-2a2fc5de7e51 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.646805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5ee09-35a1-4fbc-a78f-da5b3aa3fde7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.686818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65407c8-7ae8-412c-bde0-4f6cfd2d72ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.687466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33936a7a-79c4-43a0-8e38-edafd2e4be39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.687580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65407c8-7ae8-412c-bde0-4f6cfd2d72ad None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.687929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65407c8-7ae8-412c-bde0-4f6cfd2d72ad None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.688068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33936a7a-79c4-43a0-8e38-edafd2e4be39 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.688211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65407c8-7ae8-412c-bde0-4f6cfd2d72ad None admin] PATH_INFO: `/v3/projects/c5f7bf50f5a94c7694720e6be18c9de2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.689930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33936a7a-79c4-43a0-8e38-edafd2e4be39 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.690046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65407c8-7ae8-412c-bde0-4f6cfd2d72ad None admin] RBAC: Authorizing `identity:get_project(project_id=c5f7bf50f5a94c7694720e6be18c9de2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:46.690132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33936a7a-79c4-43a0-8e38-edafd2e4be39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.691937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65407c8-7ae8-412c-bde0-4f6cfd2d72ad None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:46.696937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 168/330] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:46 2022] GET /identity/v3/projects/c5f7bf50f5a94c7694720e6be18c9de2 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:46.700007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65407c8-7ae8-412c-bde0-4f6cfd2d72ad None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.763806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33936a7a-79c4-43a0-8e38-edafd2e4be39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:46.765321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33936a7a-79c4-43a0-8e38-edafd2e4be39 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:46.800373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65407c8-7ae8-412c-bde0-4f6cfd2d72ad None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.822135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:46.833174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779872a-6844-41a2-8813-fa69e64bf98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.833795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779872a-6844-41a2-8813-fa69e64bf98c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.834074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779872a-6844-41a2-8813-fa69e64bf98c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.834375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779872a-6844-41a2-8813-fa69e64bf98c None admin] PATH_INFO: `/v3/groups/d5bec7ecc51a45a3bbd92531325495bf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.835461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6779872a-6844-41a2-8813-fa69e64bf98c None admin] RBAC: Authorizing `identity:get_group(group_id=d5bec7ecc51a45a3bbd92531325495bf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:46.837017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6779872a-6844-41a2-8813-fa69e64bf98c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:46.838769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 169/331] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Oct 24 13:07:46 2022] GET /identity/v3/groups/d5bec7ecc51a45a3bbd92531325495bf => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:46.844815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779872a-6844-41a2-8813-fa69e64bf98c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.847434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:46.863939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 163/332] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:07:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 13:07:46.870079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33936a7a-79c4-43a0-8e38-edafd2e4be39 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:46.943068 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779872a-6844-41a2-8813-fa69e64bf98c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.964770 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33936a7a-79c4-43a0-8e38-edafd2e4be39 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:46.975313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.975910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.976191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.976496 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:46.977213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] RBAC: Adding query filter params (group.id=d5bec7ecc51a45a3bbd92531325495bf, role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=c5f7bf50f5a94c7694720e6be18c9de2) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:46.977525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:46.979902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:46.985497 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 170/333] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Oct 24 13:07:46 2022] GET /identity/v3/role_assignments?group.id=d5bec7ecc51a45a3bbd92531325495bf&scope.project.id=c5f7bf50f5a94c7694720e6be18c9de2&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:46.998674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52bdd2-d933-4ef6-89d2-fc38250b2267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:46.999320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52bdd2-d933-4ef6-89d2-fc38250b2267 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:46.999596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52bdd2-d933-4ef6-89d2-fc38250b2267 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:46.999869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52bdd2-d933-4ef6-89d2-fc38250b2267 None admin] PATH_INFO: `/v3/projects/312738b3aa6f46e2bab330e5c745742d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:47.001060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52bdd2-d933-4ef6-89d2-fc38250b2267 None admin] RBAC: Authorizing `identity:get_project(project_id=312738b3aa6f46e2bab330e5c745742d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:47.002529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52bdd2-d933-4ef6-89d2-fc38250b2267 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:47.004232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 164/334] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:07:46 2022] GET /identity/v3/projects/312738b3aa6f46e2bab330e5c745742d => generated 302 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:47.010097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:47.078579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:47.115871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc5f2-048f-4c61-a560-4a727cc708e0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:47.151546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4855203-a997-433e-8608-931793d7705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:47.152180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855203-a997-433e-8608-931793d7705b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:47.152494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855203-a997-433e-8608-931793d7705b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:47.152767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4855203-a997-433e-8608-931793d7705b None admin] PATH_INFO: `/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:47.153958 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4855203-a997-433e-8608-931793d7705b None admin] RBAC: Authorizing `identity:get_group(group_id=a750fcc292524a33b8ffb95f32a2ba6c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:47.155415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4855203-a997-433e-8608-931793d7705b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:47.157059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 171/335] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Oct 24 13:07:47 2022] GET /identity/v3/groups/a750fcc292524a33b8ffb95f32a2ba6c => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:47.163132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52bdd2-d933-4ef6-89d2-fc38250b2267 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:47.194361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:47.265469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52bdd2-d933-4ef6-89d2-fc38250b2267 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:47.298727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c757ca-3689-488a-bf2a-2ce3f3907a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:47.299318 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c757ca-3689-488a-bf2a-2ce3f3907a62 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:47.299599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c757ca-3689-488a-bf2a-2ce3f3907a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:47.299875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c757ca-3689-488a-bf2a-2ce3f3907a62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:47.300622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c757ca-3689-488a-bf2a-2ce3f3907a62 None admin] RBAC: Adding query filter params (group.id=a750fcc292524a33b8ffb95f32a2ba6c, role.id=23a3d7681bf247468a0baa9bfed6f4e9, scope.project.id=312738b3aa6f46e2bab330e5c745742d) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:47.300933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c757ca-3689-488a-bf2a-2ce3f3907a62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:47.302354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c757ca-3689-488a-bf2a-2ce3f3907a62 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:47.307871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 165/336] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Oct 24 13:07:47 2022] GET /identity/v3/role_assignments?group.id=a750fcc292524a33b8ffb95f32a2ba6c&scope.project.id=312738b3aa6f46e2bab330e5c745742d&role.id=23a3d7681bf247468a0baa9bfed6f4e9 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:50.058620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7051c0-3335-435b-a201-0a0bf0ff2d67 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:50.059665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7051c0-3335-435b-a201-0a0bf0ff2d67 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:50.060143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7051c0-3335-435b-a201-0a0bf0ff2d67 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:50.061846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 172/337] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:50.068222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62cff-e388-4f5c-8bf3-026d9fd6dea1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:50.069213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62cff-e388-4f5c-8bf3-026d9fd6dea1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:50.069706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62cff-e388-4f5c-8bf3-026d9fd6dea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:50.090024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-23e62cff-e388-4f5c-8bf3-026d9fd6dea1 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:50.091650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-23e62cff-e388-4f5c-8bf3-026d9fd6dea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:50.180564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 166/338] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:50.340496 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a695c-8bb9-49fc-9c6b-d01393afffed None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:50.340789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a695c-8bb9-49fc-9c6b-d01393afffed None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:50.340954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a695c-8bb9-49fc-9c6b-d01393afffed None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:50.342422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 173/339] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:50.348032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed673870-4879-4cf3-9281-818ebf035cef None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:50.348339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed673870-4879-4cf3-9281-818ebf035cef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:50.348622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed673870-4879-4cf3-9281-818ebf035cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:50.370880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ed673870-4879-4cf3-9281-818ebf035cef None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:50.377270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ed673870-4879-4cf3-9281-818ebf035cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:50.394055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:50.458738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 167/340] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:50.467534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8677d-4246-484b-a639-149231464b82 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:50.468055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8677d-4246-484b-a639-149231464b82 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:50.468401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8677d-4246-484b-a639-149231464b82 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:50.469980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 174/341] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:50.475695 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b007066-a157-47ef-be90-c1ea76308041 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:50.475992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b007066-a157-47ef-be90-c1ea76308041 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:50.476283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b007066-a157-47ef-be90-c1ea76308041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:50.501660 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4b007066-a157-47ef-be90-c1ea76308041 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:50.502838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4b007066-a157-47ef-be90-c1ea76308041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:50.593752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 168/342] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:50.632444 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8677d-4246-484b-a639-149231464b82 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:50.764994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8677d-4246-484b-a639-149231464b82 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:50.797742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b007066-a157-47ef-be90-c1ea76308041 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:50.825790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:50.826433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:50.826711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:50.826978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:50.840119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:50.848843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:50.850249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:50.880251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:50.881639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:50.882514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:50.882514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:50.882514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:50.882514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 175/343] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:50.888148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:50.984963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:51.014301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b007066-a157-47ef-be90-c1ea76308041 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:51.023929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c19fc-83cf-4c94-9dd0-8e12da18d19a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:51.050314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545708ae-5f66-4925-af1a-67122b74130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:51.050937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545708ae-5f66-4925-af1a-67122b74130c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.051213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545708ae-5f66-4925-af1a-67122b74130c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.051480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545708ae-5f66-4925-af1a-67122b74130c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.059738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:51.060364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.060705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.060976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.061602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:51.061942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:51.062822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545708ae-5f66-4925-af1a-67122b74130c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:51.063317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:51.064250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545708ae-5f66-4925-af1a-67122b74130c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:51.075470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 176/344] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-545708ae-5f66-4925-af1a-67122b74130c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:51.078963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:51.080176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:51.081115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:51.081115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:51.081115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:51.081115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 169/345] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:51.081367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:51.081922 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f0020-a4e0-4c52-9c23-6fe497ec324a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.082104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f0020-a4e0-4c52-9c23-6fe497ec324a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.082378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f0020-a4e0-4c52-9c23-6fe497ec324a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.087376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 170/346] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:51.087376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f0020-a4e0-4c52-9c23-6fe497ec324a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:51.139034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:51.197194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f0020-a4e0-4c52-9c23-6fe497ec324a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:51.201245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059f621-3106-4830-bde8-62bb40a474f2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:51.233655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408aa322-a61d-4be2-a95c-ea926462252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:51.234251 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aa322-a61d-4be2-a95c-ea926462252e None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.234526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aa322-a61d-4be2-a95c-ea926462252e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.234809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aa322-a61d-4be2-a95c-ea926462252e None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.235425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408aa322-a61d-4be2-a95c-ea926462252e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:51.235866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:51.236512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.236794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.236886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408aa322-a61d-4be2-a95c-ea926462252e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:51.237064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.247026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:51.248372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:51.249808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:51.264640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:51.276737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:51.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:51.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:51.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:51.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 177/347] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:51.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:51.359886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 171/348] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:51 2022] POST /identity/v3/users => generated 246 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:51.361750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8ed70-de42-40a9-8fec-d031c8a5a00e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.362217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8ed70-de42-40a9-8fec-d031c8a5a00e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.362502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8ed70-de42-40a9-8fec-d031c8a5a00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.364147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:51.367906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd28d-7454-4f85-a5ad-58e4eb2a0eff None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:51.391185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-20d8ed70-de42-40a9-8fec-d031c8a5a00e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:51.393796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-20d8ed70-de42-40a9-8fec-d031c8a5a00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:51.412634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:51.413322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.413610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.413884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.414542 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:51.414857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:51.417389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:51.434473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 178/349] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:51.437404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:51.496497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 172/350] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:51.497648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8ed70-de42-40a9-8fec-d031c8a5a00e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:51.570170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f80cf-f2ff-49e7-8c67-55941c9bebd9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:51.613504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410757a4-3e7e-4eea-852d-ad7235abd1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:51.616090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410757a4-3e7e-4eea-852d-ad7235abd1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.616090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410757a4-3e7e-4eea-852d-ad7235abd1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.616090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410757a4-3e7e-4eea-852d-ad7235abd1c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.616090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410757a4-3e7e-4eea-852d-ad7235abd1c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:51.616090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410757a4-3e7e-4eea-852d-ad7235abd1c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:51.616467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8ed70-de42-40a9-8fec-d031c8a5a00e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:51.616923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410757a4-3e7e-4eea-852d-ad7235abd1c0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:51.631270 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 179/351] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:51.633485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a7f4-8e09-41e7-aa94-e848fa217b4d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.639313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a7f4-8e09-41e7-aa94-e848fa217b4d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.639313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a7f4-8e09-41e7-aa94-e848fa217b4d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.639313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 180/352] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:51.640216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948a7f4-8e09-41e7-aa94-e848fa217b4d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:51.647261 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:51.664300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c517e23-3bcb-4a26-b8dc-fea75bd954c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:51.664300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c517e23-3bcb-4a26-b8dc-fea75bd954c2 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.664300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c517e23-3bcb-4a26-b8dc-fea75bd954c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.664300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c517e23-3bcb-4a26-b8dc-fea75bd954c2 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.664878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c517e23-3bcb-4a26-b8dc-fea75bd954c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:51.666772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c517e23-3bcb-4a26-b8dc-fea75bd954c2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:51.666938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:51.744017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 173/353] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:51 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:51.756300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9f65b-11de-40d0-8a95-a65ef5eaf2b1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.756300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9f65b-11de-40d0-8a95-a65ef5eaf2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.756300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9f65b-11de-40d0-8a95-a65ef5eaf2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.768306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948a7f4-8e09-41e7-aa94-e848fa217b4d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:51.801374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-55a9f65b-11de-40d0-8a95-a65ef5eaf2b1 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:51.802589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-55a9f65b-11de-40d0-8a95-a65ef5eaf2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:51.806265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd7f91-9cdb-4993-b7fc-9f3c93e5c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:51.806880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd7f91-9cdb-4993-b7fc-9f3c93e5c68f None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:51.807162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd7f91-9cdb-4993-b7fc-9f3c93e5c68f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:51.807439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd7f91-9cdb-4993-b7fc-9f3c93e5c68f None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:51.808110 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd7f91-9cdb-4993-b7fc-9f3c93e5c68f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:51.810092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd7f91-9cdb-4993-b7fc-9f3c93e5c68f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:51.891149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 174/354] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:51.892053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9f65b-11de-40d0-8a95-a65ef5eaf2b1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:51.924411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 181/355] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:51 2022] POST /identity/v3/users => generated 248 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:52.028469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9f65b-11de-40d0-8a95-a65ef5eaf2b1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:52.036793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:52.066630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:52.067345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:52.067634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:52.067903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:52.079293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:52.085351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:52.107626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:52.108653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:52.109938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:52.109938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:52.109938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:52.109938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 175/356] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:52.115408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd7f91-9cdb-4993-b7fc-9f3c93e5c68f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:52.210505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:52.223615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:52.245153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd7f91-9cdb-4993-b7fc-9f3c93e5c68f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:52.290812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:52.291676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:52.292425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:52.292711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:52.293676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:52.293985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:52.296062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:52.312363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 182/357] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:52.319002 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:52.353229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a12625-0cf1-4557-a2ce-d431186fb53f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:52.399519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:52.400360 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:52.400648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:52.400921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:52.402110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:52.403779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:52.442329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a1b8-0c69-4f0a-a950-7de355c80981 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:52.445143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.sql.core [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:07:52.445143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 24 13:07:52.445143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [parameters: {'id': 'ebb1f254cdbf4a08b68e7ec280dd816b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 24 13:07:52.445143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Oct 24 13:07:52.458685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return ref.to_dict() Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:07:52.459749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application yield resource Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise self.value Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application session.commit() Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:07:52.461729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:07:52.463789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:07:52.471800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:07:52.481384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [parameters: {'id': 'ebb1f254cdbf4a08b68e7ec280dd816b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:52.482684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 24 13:07:52.484377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:52.486936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 176/358] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Oct 24 13:07:52 2022] POST /identity/v3/roles => generated 144 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:52.486936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:52.490434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994d79d-6093-40de-ad68-c11e20be9ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:52.492328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994d79d-6093-40de-ad68-c11e20be9ef5 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:52.492328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994d79d-6093-40de-ad68-c11e20be9ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:52.492328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994d79d-6093-40de-ad68-c11e20be9ef5 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:52.492328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994d79d-6093-40de-ad68-c11e20be9ef5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:52.494618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994d79d-6093-40de-ad68-c11e20be9ef5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:52.504525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:52.523077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:52.605419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 183/359] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:52 2022] POST /identity/v3/users => generated 249 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:52.614853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5189c0-a615-4020-b50a-b47f0024c495 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:52.653680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3bb9f-726e-4b91-8716-63125e91a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:52.654333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3bb9f-726e-4b91-8716-63125e91a060 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:52.654617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3bb9f-726e-4b91-8716-63125e91a060 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:52.654955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3bb9f-726e-4b91-8716-63125e91a060 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:52.667887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3bb9f-726e-4b91-8716-63125e91a060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5ab3bb9f-726e-4b91-8716-63125e91a060 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:52.673183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:52.674235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:52.675447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 177/360] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:07:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:52.680711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994d79d-6093-40de-ad68-c11e20be9ef5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:52.830138 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994d79d-6093-40de-ad68-c11e20be9ef5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:52.871353 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5ad03-5b83-4313-93ca-3dc1883cfdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:52.871977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5ad03-5b83-4313-93ca-3dc1883cfdce None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:52.872255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5ad03-5b83-4313-93ca-3dc1883cfdce None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:52.872559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5ad03-5b83-4313-93ca-3dc1883cfdce None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:52.873244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5ad03-5b83-4313-93ca-3dc1883cfdce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:52.873547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5ad03-5b83-4313-93ca-3dc1883cfdce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:52.874942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5ad03-5b83-4313-93ca-3dc1883cfdce None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:52.884942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 184/361] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:53.752125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e491-69b9-4b82-bb4d-9d1bcc3b9993 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:53.752792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e491-69b9-4b82-bb4d-9d1bcc3b9993 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:53.752792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7e491-69b9-4b82-bb4d-9d1bcc3b9993 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:53.754293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 178/362] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:53.759591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977e6f3-ca46-4ee0-a8bb-0eed33133e28 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:53.760084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977e6f3-ca46-4ee0-a8bb-0eed33133e28 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:53.760242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977e6f3-ca46-4ee0-a8bb-0eed33133e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:53.782417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9977e6f3-ca46-4ee0-a8bb-0eed33133e28 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:53.783581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9977e6f3-ca46-4ee0-a8bb-0eed33133e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:53.836205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:53.865616 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 185/363] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:54.226229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7e491-69b9-4b82-bb4d-9d1bcc3b9993 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:54.333349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:54.357952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7e491-69b9-4b82-bb4d-9d1bcc3b9993 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:54.404092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92548a7-e62b-4ea8-8112-78be67845d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:54.404747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92548a7-e62b-4ea8-8112-78be67845d17 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:54.405028 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92548a7-e62b-4ea8-8112-78be67845d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:54.405299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92548a7-e62b-4ea8-8112-78be67845d17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:54.423107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92548a7-e62b-4ea8-8112-78be67845d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c92548a7-e62b-4ea8-8112-78be67845d17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:54.427952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:54.428983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:54.429849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 179/364] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:07:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:54.435757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977e6f3-ca46-4ee0-a8bb-0eed33133e28 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:54.579057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977e6f3-ca46-4ee0-a8bb-0eed33133e28 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:54.592997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3e32-4122-4067-b165-3d16d0dcb8dd None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:54.593299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3e32-4122-4067-b165-3d16d0dcb8dd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:54.593546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3e32-4122-4067-b165-3d16d0dcb8dd None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:54.595015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 180/365] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:07:54.600945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f14ceb-72b1-42e8-a26a-f7716535d176 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:54.601229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f14ceb-72b1-42e8-a26a-f7716535d176 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:54.601506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f14ceb-72b1-42e8-a26a-f7716535d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:54.623820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c4f14ceb-72b1-42e8-a26a-f7716535d176 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:54.624960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c4f14ceb-72b1-42e8-a26a-f7716535d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:54.639019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:54.639627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:54.639902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:54.640175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:54.640863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:54.641165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:54.642535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:54.661277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 186/366] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:07:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:54.667666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:54.756663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:54.757633 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 181/367] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:54.810067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7a068-f98b-46f2-8bf4-5c53a2f0eaec None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:54.810708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7a068-f98b-46f2-8bf4-5c53a2f0eaec None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:54.811200 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7a068-f98b-46f2-8bf4-5c53a2f0eaec None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:54.813033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 182/368] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:07:54.823068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b80f2-2be4-42ca-8b20-4d6de43b6948 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:54.823600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b80f2-2be4-42ca-8b20-4d6de43b6948 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:54.824096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b80f2-2be4-42ca-8b20-4d6de43b6948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:54.843051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-301b80f2-2be4-42ca-8b20-4d6de43b6948 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:54.844551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-301b80f2-2be4-42ca-8b20-4d6de43b6948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:54.848393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68378c-7ce8-4620-80d8-151c18f38d4f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:54.918441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 183/369] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:07:54.926308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:54.927191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:54.927670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:54.928146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:54.947215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:54.948994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:54.964115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:54.965097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:54.965966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:54.965966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:54.965966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:54.968011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 187/370] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:54.973791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b80f2-2be4-42ca-8b20-4d6de43b6948 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:54.976578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:55.124207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b80f2-2be4-42ca-8b20-4d6de43b6948 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.161756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.174734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.175653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.176133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.176634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.177491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:55.178217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:55.182157 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:55.198372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 184/371] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:55.199534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.268488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:55.297968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:55.303851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c352f7-cf53-4e80-904b-3adda1e18eb4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.321373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f66790-906b-43c3-b964-3396fc0d0fc7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.339059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f6f21-ff7d-4e48-a4c7-27a3f99845f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.339739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f6f21-ff7d-4e48-a4c7-27a3f99845f2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.340017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f6f21-ff7d-4e48-a4c7-27a3f99845f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.340299 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f6f21-ff7d-4e48-a4c7-27a3f99845f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.351268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f6f21-ff7d-4e48-a4c7-27a3f99845f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-518f6f21-ff7d-4e48-a4c7-27a3f99845f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:55.356875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:55.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:55.358698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 188/372] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:07:55.363834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f6f21-ff7d-4e48-a4c7-27a3f99845f2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.370627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363691b-0b23-46ac-91d8-77af833f2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.371536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363691b-0b23-46ac-91d8-77af833f2845 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.372026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363691b-0b23-46ac-91d8-77af833f2845 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.372571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363691b-0b23-46ac-91d8-77af833f2845 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.387503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363691b-0b23-46ac-91d8-77af833f2845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0363691b-0b23-46ac-91d8-77af833f2845 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:55.395409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:55.396382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:07:55.397216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:55.397864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 185/373] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:55.398648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363691b-0b23-46ac-91d8-77af833f2845 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.483307 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f6f21-ff7d-4e48-a4c7-27a3f99845f2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.530035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.530647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.530887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.531156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.533465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363691b-0b23-46ac-91d8-77af833f2845 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.541359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:55.543105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:07:55.557028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:55.558217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:07:55.559345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:55.560730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 189/374] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:55.565992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:55.566752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.580635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.581746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.583175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.583175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.584158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:55.584672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:55.586521 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:55.588911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:55.604127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 186/375] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:55.605077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.682238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849811e-8e00-4992-bf2b-9c4eed20367c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.708049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720b4eb-a471-409b-a2a8-55a85e854b99 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.715277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.715869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.716144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.716451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.717096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:55.717393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:55.719013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:55.726575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 190/376] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:55.727367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.743257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bb588-a8d9-4025-9d4c-464aa73ca1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.743797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bb588-a8d9-4025-9d4c-464aa73ca1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.744080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bb588-a8d9-4025-9d4c-464aa73ca1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.744381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bb588-a8d9-4025-9d4c-464aa73ca1f5 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.745113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bb588-a8d9-4025-9d4c-464aa73ca1f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:55.745421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bb588-a8d9-4025-9d4c-464aa73ca1f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:55.746855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bb588-a8d9-4025-9d4c-464aa73ca1f5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:55.783579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 187/377] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:55.784896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0751b36-1e22-48a6-8440-7cfca8664c4f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.785263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0751b36-1e22-48a6-8440-7cfca8664c4f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.785605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0751b36-1e22-48a6-8440-7cfca8664c4f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.786875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 188/378] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:55.787900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f2d4-ab3a-40b6-a1c0-45c5e19310ca None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.788150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f2d4-ab3a-40b6-a1c0-45c5e19310ca None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.788448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f2d4-ab3a-40b6-a1c0-45c5e19310ca None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.789913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 189/379] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:07:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:55.794531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088f2d4-ab3a-40b6-a1c0-45c5e19310ca None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.810801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:55.831611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db01754-0fd0-44e2-b53c-34f2ab950643 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.838187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:55.864789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.865408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.865692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.865962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.878646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:55.880126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:55.894706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:55.895628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:55.896575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:55.896575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:55.896575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:55.896739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088f2d4-ab3a-40b6-a1c0-45c5e19310ca None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:55.896821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 191/380] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:55.896821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:55.930716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3de12-18d8-4890-813f-524cbdebd5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:55.931341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3de12-18d8-4890-813f-524cbdebd5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.931622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3de12-18d8-4890-813f-524cbdebd5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.931896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3de12-18d8-4890-813f-524cbdebd5f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.941772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3de12-18d8-4890-813f-524cbdebd5f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:55.943231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3de12-18d8-4890-813f-524cbdebd5f6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-00c3de12-18d8-4890-813f-524cbdebd5f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:55.958314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:55.959372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:55.960419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:55.960419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:55.960419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:55.960419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 190/381] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:55.960706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1bcd-f858-42bf-a191-e98079362abf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:55.960931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1bcd-f858-42bf-a191-e98079362abf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:55.961236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1bcd-f858-42bf-a191-e98079362abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:55.979001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a6ab1bcd-f858-42bf-a191-e98079362abf None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:55.980187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a6ab1bcd-f858-42bf-a191-e98079362abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:55.995837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafd7a6-ddc4-462c-8f98-791a715dc7fc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.026778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:56.032219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.032862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.033143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.033415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.043437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.044907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.045008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 191/382] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:56.046184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475798-73e3-4765-86c7-3778e68463b9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.046472 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475798-73e3-4765-86c7-3778e68463b9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.046748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475798-73e3-4765-86c7-3778e68463b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:56.060546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:56.061538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:56.062422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:56.062422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:56.062422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:56.062422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 192/383] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:56.062422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.070067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-69475798-73e3-4765-86c7-3778e68463b9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:56.071824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-69475798-73e3-4765-86c7-3778e68463b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:56.095055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:56.156636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 192/384] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:07:56.158026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69475798-73e3-4765-86c7-3778e68463b9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.178677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a32d4f-a1d2-479a-bbb6-d39c8b0e3215 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.220603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.221699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.222552 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.223018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.223847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:56.224361 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.226729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.244326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 193/385] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:56.250230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.275946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69475798-73e3-4765-86c7-3778e68463b9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.309488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:56.315295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.316066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.316377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.316943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.317596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:56.317901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.319271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.335269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 193/386] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:56.337619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.356838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:56.369694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55048d7a-d410-4d92-a80a-77b2d718d16f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.405473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.406283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.406561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.406831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.407461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:56.407762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.409152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.421070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 194/387] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:56.421873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.447421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a9956-f959-40e2-aecf-d32d7222ebd2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.481010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.481618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.482232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.482232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.489135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.490552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:07:56.503599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:56.504716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:56.505630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 194/388] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:56.506322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.521319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fcee1-3e9c-491f-b5c4-a91497d65483 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.522845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:56.556610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ff5ad-a777-4831-968f-296a84bb5907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.557448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ff5ad-a777-4831-968f-296a84bb5907 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.557896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ff5ad-a777-4831-968f-296a84bb5907 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.558164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ff5ad-a777-4831-968f-296a84bb5907 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.567972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ff5ad-a777-4831-968f-296a84bb5907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-059ff5ad-a777-4831-968f-296a84bb5907 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:07:56.572754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:56.573798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 195/389] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:56.574788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ff5ad-a777-4831-968f-296a84bb5907 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.578048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:56.603993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750abf5-8198-47c6-b42d-6d524ff5a915 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.637704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.638317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.638595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.638867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] PATH_INFO: `/v3/users/glance` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.646087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.647521 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:07:56.658361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:56.659443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 195/390] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:56.665685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.679526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ff5ad-a777-4831-968f-296a84bb5907 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.712797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.713488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.713840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.714176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.724955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.726472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:56.741007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:56.742072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:56.743533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:56.743533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:56.743533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:56.743533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 196/391] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:56.743798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.759379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3ed4f-e352-41e7-9478-378218239bbd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.759521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:56.793000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.793615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.793895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.794169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.796018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:56.801249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.802699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:07:56.814611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:56.815595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 196/392] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:56.816712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.841819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df5a4a-4bf2-486e-a969-a4bf89083fc2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.875415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.876037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.876300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.876575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.877295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:56.877602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.879026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.914382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 197/393] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:56.915237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.916085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c75c6-d6de-4688-a31b-2b162233ead8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:56.950583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:56.951173 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:56.951450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:56.951719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:56.952410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:56.952715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:56.954074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:56.962058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 197/394] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:56.962834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:56.988549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:57.017872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffaf3e-3a46-40a6-8933-f4b374954f00 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.020180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:57.050997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.051588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.051879 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.052136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.052880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:57.053186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.054601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.062541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4257d-932e-46da-ba7e-8b68fb6ca31d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.072294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 198/395] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:57.073137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.099121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.099714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.099989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.100259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.100911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:57.101210 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.103111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.114428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 198/396] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:57.115199 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.172067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bda23b-f202-414e-9f2a-43bb37725641 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.200048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:57.205661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.206253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.206535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.210533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.210533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:57.210533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.210533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.213624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba757d-6309-47ca-b2d8-35d65c957a3d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.227709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 199/397] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:57.228733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.233305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:57.246802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.247424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.247706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.247980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.258149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.260151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:57.275430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:57.276630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:57.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:57.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:57.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:57.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 199/398] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:57.277721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.330623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ab537-dfb5-4fcd-b867-f764ce88810c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.363400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.364011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.364307 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.364589 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.374523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1450dda-d908-49bb-98c2-8f45900c9994 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.374667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.376091 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:57.391709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:57.392819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:57.393915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:57.393915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:57.393915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:57.393915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 200/399] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:57.393915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.412225 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.413015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.413295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.413572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.423317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.424783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.437110 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:57.439622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:57.447406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:57.448299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:57.448299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:57.448299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:57.448299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 200/400] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:57.448299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.469019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:57.493304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc6225-c3aa-4399-b758-9df3240d0910 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.527362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0db08-2c5f-4c05-8b1b-686f59c9f617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.527889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0db08-2c5f-4c05-8b1b-686f59c9f617 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.528168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0db08-2c5f-4c05-8b1b-686f59c9f617 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.528471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0db08-2c5f-4c05-8b1b-686f59c9f617 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.529114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0db08-2c5f-4c05-8b1b-686f59c9f617 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.530556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0db08-2c5f-4c05-8b1b-686f59c9f617 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.544881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35434-8396-4f56-b8bd-9dc886c67f1f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.577928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcd0e6-3cc9-4aa0-982a-554f12baf41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.578561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcd0e6-3cc9-4aa0-982a-554f12baf41b None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.578836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcd0e6-3cc9-4aa0-982a-554f12baf41b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.579105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcd0e6-3cc9-4aa0-982a-554f12baf41b None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/5c5b4b17788944f8a0991cb62b2a7205/roles/04a120a50aa648fea07c999a73a1c22c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.592548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcd0e6-3cc9-4aa0-982a-554f12baf41b None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=5c5b4b17788944f8a0991cb62b2a7205, role_id=04a120a50aa648fea07c999a73a1c22c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.594208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcd0e6-3cc9-4aa0-982a-554f12baf41b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.639322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 201/401] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:07:57 2022] POST /identity/v3/users => generated 247 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:07:57.640219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0db08-2c5f-4c05-8b1b-686f59c9f617 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.677652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 201/402] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:57 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/5c5b4b17788944f8a0991cb62b2a7205/roles/04a120a50aa648fea07c999a73a1c22c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:57.677652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcd0e6-3cc9-4aa0-982a-554f12baf41b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.781396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:57.794984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:57.802846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0db08-2c5f-4c05-8b1b-686f59c9f617 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.820871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcd0e6-3cc9-4aa0-982a-554f12baf41b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:57.855146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.856673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.857513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.857990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.858650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:57.858989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.861260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.868428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:57.869120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:57.869468 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:57.869807 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:57.870516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:57.870896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:57.872358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:57.877350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 202/403] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:57.878945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:57.894493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 202/404] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:57.895578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.002069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d2856-f87d-4b44-8018-cd681ee0f52b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.005427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c0946-e470-4e04-a840-05fa3f25eed2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.037508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:58.040674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.041278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.041558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.041826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.042459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:58.042763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.044384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.050014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.050128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.050128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.050128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.052850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.054386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.057343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 203/405] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:58.058249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:07:58.073951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:58.074883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:58.075749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 203/406] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:58.076533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.094107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:58.185347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260ba9b-12c5-4e9b-b06d-1321bb543c7f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.191789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6aea1-58bd-46a0-9251-f04ef7cd574b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.222876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.223857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.224319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.224729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.227934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.228671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.228861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.229135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.234111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.235762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.236002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.237663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:07:58.248833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:58.249938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:07:58.251049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:07:58.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:58.252368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:58.253313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:07:58.253313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:07:58.253313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:07:58.253313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:07:58.253313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:07:58.253313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:07:58.253313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:07:58.253313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:58.259550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 204/407] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:58.259550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.260114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 204/408] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:07:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:58.261148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.308022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:58.354814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:58.359906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05472e5-034a-4ec0-97bf-cdbd06bed2c2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.379573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56982b-5da5-40d2-a705-caabb5b9cb89 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.396359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.397186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.398142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.398605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.400359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:58.400849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.402444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.419008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.419734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.420087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.420888 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.421905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:58.422412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.423986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.429191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 205/409] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:58.429831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.453756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 205/410] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Oct 24 13:07:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:58.453756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.557346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673d5c3-bb50-4354-8993-6db382633b83 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.562192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67190b7-7d63-42a0-9a31-65cffa9971ab None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.574961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:58.594604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166cde5-d4ae-4c55-ab1e-919fc84c7fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.595289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166cde5-d4ae-4c55-ab1e-919fc84c7fc3 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.595571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166cde5-d4ae-4c55-ab1e-919fc84c7fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.596380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166cde5-d4ae-4c55-ab1e-919fc84c7fc3 None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/32fa3c94cad04287aedea87400a15591/roles/04a120a50aa648fea07c999a73a1c22c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.601370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:58.607464 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.608450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.608930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.609503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.617980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:58.617980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.617980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.618320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166cde5-d4ae-4c55-ab1e-919fc84c7fc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=32fa3c94cad04287aedea87400a15591, role_id=04a120a50aa648fea07c999a73a1c22c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.618526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166cde5-d4ae-4c55-ab1e-919fc84c7fc3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.626757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 206/411] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:58.627857 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.687930 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 206/412] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:58 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/32fa3c94cad04287aedea87400a15591/roles/04a120a50aa648fea07c999a73a1c22c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:58.689057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166cde5-d4ae-4c55-ab1e-919fc84c7fc3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.811635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4144a-b158-43c5-9ada-d1f3d44aa654 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.890016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.890619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166cde5-d4ae-4c55-ab1e-919fc84c7fc3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:58.890964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.890964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.891114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.902752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.904215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:07:58.922694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:58.923669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:07:58.924565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:07:58.924565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:07:58.924565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:07:58.924565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 207/413] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:58.931066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:58.934430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d16b4-33c5-40d4-83be-af4dee62ee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:58.935092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d16b4-33c5-40d4-83be-af4dee62ee0a None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:58.935345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d16b4-33c5-40d4-83be-af4dee62ee0a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:58.935617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d16b4-33c5-40d4-83be-af4dee62ee0a None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/6ee2800cf7f744a8973d5c5ca3907af1/roles/04a120a50aa648fea07c999a73a1c22c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:58.955455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d16b4-33c5-40d4-83be-af4dee62ee0a None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=6ee2800cf7f744a8973d5c5ca3907af1, role_id=04a120a50aa648fea07c999a73a1c22c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:58.957119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d16b4-33c5-40d4-83be-af4dee62ee0a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:58.969813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:59.033329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 207/414] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:58 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/6ee2800cf7f744a8973d5c5ca3907af1/roles/04a120a50aa648fea07c999a73a1c22c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:07:59.060307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4bc26-faa9-40c0-aa78-67082fa395e1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:59.106618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:59.107235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:59.107517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:59.107785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:59.108463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:59.108768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:59.110171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:59.122778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 208/415] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:07:59.129372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d16b4-33c5-40d4-83be-af4dee62ee0a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:59.181419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:59.258816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d16b4-33c5-40d4-83be-af4dee62ee0a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:59.296416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:59.297050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:59.297331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:59.297604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:59.304929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:59.306525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:07:59.327099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:07:59.328038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:07:59.328939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 208/416] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:07:59.334483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:59.424791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:59.450661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff5452-659c-4a69-83c7-7998cdf050b0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:59.480403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091ffcb-60e5-46fe-91c7-db141426b4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:59.481016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ffcb-60e5-46fe-91c7-db141426b4fc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:59.481296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ffcb-60e5-46fe-91c7-db141426b4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:59.481571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091ffcb-60e5-46fe-91c7-db141426b4fc None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:59.482280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091ffcb-60e5-46fe-91c7-db141426b4fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:07:59.482562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091ffcb-60e5-46fe-91c7-db141426b4fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:59.483986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091ffcb-60e5-46fe-91c7-db141426b4fc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:59.496720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 209/417] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:07:59.502297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:59.612689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:59.622415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528bfee-aba7-4be1-a939-bf6339b1b96a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:59.671454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2caedb-604d-4654-8942-ad910f40dedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:59.672112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2caedb-604d-4654-8942-ad910f40dedb None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:59.672464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2caedb-604d-4654-8942-ad910f40dedb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:59.672741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2caedb-604d-4654-8942-ad910f40dedb None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/65d4ca1062fe496dbe16d6cd1bcac1a8/roles/04a120a50aa648fea07c999a73a1c22c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:59.692532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2caedb-604d-4654-8942-ad910f40dedb None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=65d4ca1062fe496dbe16d6cd1bcac1a8, role_id=04a120a50aa648fea07c999a73a1c22c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:59.693985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2caedb-604d-4654-8942-ad910f40dedb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:59.782808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 209/418] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:07:59 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/65d4ca1062fe496dbe16d6cd1bcac1a8/roles/04a120a50aa648fea07c999a73a1c22c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:08:00.016312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf795b98-6a15-4f8e-b4ae-2890337d0e4b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:00.016312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf795b98-6a15-4f8e-b4ae-2890337d0e4b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:00.016312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf795b98-6a15-4f8e-b4ae-2890337d0e4b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:00.017851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 210/419] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:00.023921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14812-b524-4db9-b034-b212c2dd4741 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:00.024947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14812-b524-4db9-b034-b212c2dd4741 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:00.025414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a14812-b524-4db9-b034-b212c2dd4741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:00.045037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-55a14812-b524-4db9-b034-b212c2dd4741 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:00.046547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-55a14812-b524-4db9-b034-b212c2dd4741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:00.085158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a111-329a-4fb9-b311-abcbedabb405 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:00.085778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a111-329a-4fb9-b311-abcbedabb405 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:00.086243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a111-329a-4fb9-b311-abcbedabb405 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:00.088019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 211/420] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:00.098384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f818a7-b8f1-45a8-8ce7-7234eea8e006 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:00.098903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f818a7-b8f1-45a8-8ce7-7234eea8e006 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:00.099408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f818a7-b8f1-45a8-8ce7-7234eea8e006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:00.120713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-05f818a7-b8f1-45a8-8ce7-7234eea8e006 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:00.122228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-05f818a7-b8f1-45a8-8ce7-7234eea8e006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:00.128879 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 210/421] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:00.204607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 212/422] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:08:00.527876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a14812-b524-4db9-b034-b212c2dd4741 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:00.544825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f818a7-b8f1-45a8-8ce7-7234eea8e006 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:00.644642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:00.661276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:00.700986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a14812-b524-4db9-b034-b212c2dd4741 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:00.704395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f818a7-b8f1-45a8-8ce7-7234eea8e006 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:00.752539 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd81f2-746f-4592-b13f-bc9013131185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:00.753164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd81f2-746f-4592-b13f-bc9013131185 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:00.753472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd81f2-746f-4592-b13f-bc9013131185 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:00.753745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd81f2-746f-4592-b13f-bc9013131185 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:00.765604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd81f2-746f-4592-b13f-bc9013131185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-62bd81f2-746f-4592-b13f-bc9013131185 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:00.770518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:00.771436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:00.772239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 211/423] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:00.776061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d0eea-863d-425e-abd0-cf31c64dfb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:00.776773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d0eea-863d-425e-abd0-cf31c64dfb80 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:00.777109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d0eea-863d-425e-abd0-cf31c64dfb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:00.777439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d0eea-863d-425e-abd0-cf31c64dfb80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:00.780367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd81f2-746f-4592-b13f-bc9013131185 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:00.810443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d0eea-863d-425e-abd0-cf31c64dfb80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b16d0eea-863d-425e-abd0-cf31c64dfb80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:00.816325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:00.817224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:00.818035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:00.824631 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 213/424] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:00.830679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d0eea-863d-425e-abd0-cf31c64dfb80 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:00.954783 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd81f2-746f-4592-b13f-bc9013131185 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.019909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.020544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.020819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.021085 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.021754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:01.022054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.023412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.029032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d0eea-863d-425e-abd0-cf31c64dfb80 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.029970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:01.034696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 212/425] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:08:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:01.040381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:01.103179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:01.113799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.114425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.114763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.115031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.115752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:01.116057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.119174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.128800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 214/426] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:08:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:01.135128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:01.219344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e92e8-4377-418a-8ab9-00aa1a4eba63 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.257721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91f277-f9ba-4dc4-8fa5-46b8c36ec0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.258424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91f277-f9ba-4dc4-8fa5-46b8c36ec0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.258701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91f277-f9ba-4dc4-8fa5-46b8c36ec0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.259014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91f277-f9ba-4dc4-8fa5-46b8c36ec0b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.269306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91f277-f9ba-4dc4-8fa5-46b8c36ec0b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.270806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91f277-f9ba-4dc4-8fa5-46b8c36ec0b9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.273669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c49f-5c19-4a4a-bdf7-6ddbe2dfcfeb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bb91f277-f9ba-4dc4-8fa5-46b8c36ec0b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:01.286089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:01.286998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:01.287856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:01.287856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:01.287856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:01.287856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 213/427] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:01.292292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7f2f5-a84e-4d70-aa1e-3bc846c53baa None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.292292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7f2f5-a84e-4d70-aa1e-3bc846c53baa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.292292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7f2f5-a84e-4d70-aa1e-3bc846c53baa None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.292292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 214/428] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:01.294731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7f2f5-a84e-4d70-aa1e-3bc846c53baa None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:01.323725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31924f-6c6b-4f6f-ba5f-1381f62018c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.324859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31924f-6c6b-4f6f-ba5f-1381f62018c1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.325191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31924f-6c6b-4f6f-ba5f-1381f62018c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.325740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31924f-6c6b-4f6f-ba5f-1381f62018c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.341813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31924f-6c6b-4f6f-ba5f-1381f62018c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.344031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31924f-6c6b-4f6f-ba5f-1381f62018c1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.354995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6b31924f-6c6b-4f6f-ba5f-1381f62018c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:01.365949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:01.367296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:01.368991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:01.368991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:01.368991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:01.368991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 215/429] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:01.368991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d215f8-b548-4ac4-b4af-532f4b90a945 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.369454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d215f8-b548-4ac4-b4af-532f4b90a945 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.372611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d215f8-b548-4ac4-b4af-532f4b90a945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.390079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-52d215f8-b548-4ac4-b4af-532f4b90a945 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:01.395316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-52d215f8-b548-4ac4-b4af-532f4b90a945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:01.413028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7f2f5-a84e-4d70-aa1e-3bc846c53baa None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.446236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:01.459588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.460188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.462112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.462380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.463019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:01.463312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.464697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.478244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 215/430] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:01.484293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:01.484859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 216/431] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:01.486087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d215f8-b548-4ac4-b4af-532f4b90a945 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:01.597129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d215f8-b548-4ac4-b4af-532f4b90a945 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.644297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.645358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.645840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.646338 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.654874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.656573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.663462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bea3d9-5659-4b7d-a3b8-e1aa0719deac None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:01.670022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:01.670938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:08:01.671849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:01.674371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 217/432] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:01.675030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:01.685259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:01.710431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.711023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.711336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.712179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.712850 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:01.713153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.716166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.742899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 216/433] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:01.744762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:01.756110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:01.860376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c9692-8d42-48fc-9a63-a9c5e82dc184 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.903650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.909614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.909614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.909614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.909848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82b152-68df-494c-ae7b-6c5b60b85ffd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:01.913134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.914607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:01.927079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:01.928204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:01.929360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:01.932364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 217/434] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:01.933347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:01.945206 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e7fc1-e4b2-4e31-8cd6-56790137283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:01.952584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e7fc1-e4b2-4e31-8cd6-56790137283c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.952584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e7fc1-e4b2-4e31-8cd6-56790137283c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.952584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e7fc1-e4b2-4e31-8cd6-56790137283c None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.952584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e7fc1-e4b2-4e31-8cd6-56790137283c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:01.952584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e7fc1-e4b2-4e31-8cd6-56790137283c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:01.952584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e7fc1-e4b2-4e31-8cd6-56790137283c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:01.965826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:01.968395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 218/435] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:01.969557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34062e57-4a8d-4848-b9eb-e7d43b89173e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:01.969842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34062e57-4a8d-4848-b9eb-e7d43b89173e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:01.970109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34062e57-4a8d-4848-b9eb-e7d43b89173e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:01.975473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 219/436] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:01.975473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34062e57-4a8d-4848-b9eb-e7d43b89173e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.047757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83075362-9cc0-4860-8b53-ccadf80508ac None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.074158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:02.084644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34062e57-4a8d-4848-b9eb-e7d43b89173e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.086810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19162e6f-56b3-4daa-a0d3-62be480ec415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.087436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19162e6f-56b3-4daa-a0d3-62be480ec415 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.087699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19162e6f-56b3-4daa-a0d3-62be480ec415 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.087975 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19162e6f-56b3-4daa-a0d3-62be480ec415 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.101793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19162e6f-56b3-4daa-a0d3-62be480ec415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-19162e6f-56b3-4daa-a0d3-62be480ec415 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:02.106663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:02.107573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:02.108435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 218/437] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:02.109107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19162e6f-56b3-4daa-a0d3-62be480ec415 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.126850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2d29e-9b67-4248-a01f-e441e6f1c2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.127382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2d29e-9b67-4248-a01f-e441e6f1c2e9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.127658 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2d29e-9b67-4248-a01f-e441e6f1c2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.127921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2d29e-9b67-4248-a01f-e441e6f1c2e9 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.128655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2d29e-9b67-4248-a01f-e441e6f1c2e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:02.129123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2d29e-9b67-4248-a01f-e441e6f1c2e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:02.130840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b2d29e-9b67-4248-a01f-e441e6f1c2e9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:02.148765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 220/438] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Oct 24 13:08:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:02.150052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450547-72c4-4a06-91d4-377477261780 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.150335 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450547-72c4-4a06-91d4-377477261780 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.150606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450547-72c4-4a06-91d4-377477261780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.168574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-13450547-72c4-4a06-91d4-377477261780 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:02.169744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-13450547-72c4-4a06-91d4-377477261780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:02.204751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:02.214014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19162e6f-56b3-4daa-a0d3-62be480ec415 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.256294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 221/439] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:02.256294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13450547-72c4-4a06-91d4-377477261780 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.259407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.260115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.260480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.260818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.272459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:02.274077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:02.291731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:02.292662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:02.293672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:02.293672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:02.293672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:02.293672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 219/440] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:02.294309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.355558 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:02.375002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13450547-72c4-4a06-91d4-377477261780 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.411466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.412465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.412915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.413378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.414206 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:02.415234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:02.416870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:02.425851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 222/441] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:02.426844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.433847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ec146-5731-4307-9900-fc9549cbfaf6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.470327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d992-4554-4327-a117-facdf139688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.470929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d992-4554-4327-a117-facdf139688d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.471210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d992-4554-4327-a117-facdf139688d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.471485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d992-4554-4327-a117-facdf139688d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.509174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:02.535568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949ad64-d3f4-4847-b453-f82c3a7c44fb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.569113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3d992-4554-4327-a117-facdf139688d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:02.569658 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.570249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.570523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.570619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3d992-4554-4327-a117-facdf139688d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:02.570789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.571407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:02.571705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:02.573077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:02.584798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 223/442] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:02.585591 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.589067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:02.672163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 220/443] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:08:02.673804 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7e6b9-73d7-401a-98f3-ad9c81ecc2ac None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.674090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7e6b9-73d7-401a-98f3-ad9c81ecc2ac None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.674359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7e6b9-73d7-401a-98f3-ad9c81ecc2ac None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.675695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 221/444] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:02.676411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7e6b9-73d7-401a-98f3-ad9c81ecc2ac None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.685813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16765d-387f-4ef1-8a9f-edf3c363c16e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.718326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.718939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.719220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.719487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.721257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:02.730828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:02.732297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:02.746760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:02.747703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:02.748578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:02.748578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:02.748578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:02.748578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 224/445] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:02.748808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.778167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7e6b9-73d7-401a-98f3-ad9c81ecc2ac None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.803861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:02.813596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c43143-dbab-4377-9877-15538b256e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.815441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c43143-dbab-4377-9877-15538b256e07 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.815441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c43143-dbab-4377-9877-15538b256e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.815441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c43143-dbab-4377-9877-15538b256e07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.825009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c43143-dbab-4377-9877-15538b256e07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-67c43143-dbab-4377-9877-15538b256e07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:02.829650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:02.830631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:02.831474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 222/446] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:02.832202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c43143-dbab-4377-9877-15538b256e07 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.849108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1482f87-94cf-4bae-9ba5-7545ed1b99c7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.882968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ab972-cf48-4000-88fc-0b3cbfdab9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.884159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ab972-cf48-4000-88fc-0b3cbfdab9ff None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.884474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ab972-cf48-4000-88fc-0b3cbfdab9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.884750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ab972-cf48-4000-88fc-0b3cbfdab9ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.891227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ab972-cf48-4000-88fc-0b3cbfdab9ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:02.892682 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ab972-cf48-4000-88fc-0b3cbfdab9ff None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ed8ab972-cf48-4000-88fc-0b3cbfdab9ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:02.903617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:02.904490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:02.905353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 225/447] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:02.905961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19aa3c-b038-423c-a72d-fcf12ac84a79 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.906105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19aa3c-b038-423c-a72d-fcf12ac84a79 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.906962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19aa3c-b038-423c-a72d-fcf12ac84a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.923927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6f19aa3c-b038-423c-a72d-fcf12ac84a79 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:02.925096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6f19aa3c-b038-423c-a72d-fcf12ac84a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:02.931278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c43143-dbab-4377-9877-15538b256e07 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:02.933642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:02.965201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:02.965814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:02.966090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:02.966360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:02.976357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:02.977827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:02.989812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 226/448] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:02.990622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19aa3c-b038-423c-a72d-fcf12ac84a79 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:02.991933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:02.993532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:02.994431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:02.994431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:02.994431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:02.994431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 223/449] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:02.994431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.048849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:03.107302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae7865-580d-47e4-92ac-b9e56d5b7150 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.110724 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19aa3c-b038-423c-a72d-fcf12ac84a79 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.141741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.142333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.142608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.142878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.143529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:03.143827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.145207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.147875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.148497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.148778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.149586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.150215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:03.150518 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.151859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.158743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 224/450] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:03.159301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.163754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 227/451] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:03.164455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.182144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:03.267041 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b973a7-454d-4be2-81ef-d26a30a238b7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.270308 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ae0c0-e8c6-4548-88a0-7f94a1b82dcb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.277778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:03.300406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.300994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.301273 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.301544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.302226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:03.302530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.303046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.303634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.303911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.303997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.304181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.304834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:03.305158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.306595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.315081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 225/452] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:03.315868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.318518 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 228/453] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:03.320520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.392079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:03.414007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f564ca-18f9-4dae-9dfa-7fc4dd29e74f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.421265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6132302-fa1e-44f6-b664-b006400810aa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.446909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.447463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.447751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.448022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.454520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.455205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.455485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.455751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.458499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.459947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.463909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.465394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:03.474800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:03.475714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:03.476600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:03.476600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:03.476600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:03.476600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 226/454] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:03.477319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:03.477417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:03.478248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:03.479169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 229/455] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:03.480638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.518336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:03.587911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c582e3-8900-4e67-a3e4-a433682ded40 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.589019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a3b31-0965-4a27-961f-1c5eb23f9dd0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.613375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:03.621620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9364b-676f-4ce6-af38-b4b879db799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.622230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9364b-676f-4ce6-af38-b4b879db799a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.622331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020bc5a-5780-4ecc-83f6-555d4cdc70c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.622504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9364b-676f-4ce6-af38-b4b879db799a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.622774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9364b-676f-4ce6-af38-b4b879db799a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.622931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020bc5a-5780-4ecc-83f6-555d4cdc70c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.623215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020bc5a-5780-4ecc-83f6-555d4cdc70c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.623490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020bc5a-5780-4ecc-83f6-555d4cdc70c6 None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/035a080de9d54a56af29cc56b33a1ddc/roles/04a120a50aa648fea07c999a73a1c22c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.632930 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9364b-676f-4ce6-af38-b4b879db799a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:03.638179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020bc5a-5780-4ecc-83f6-555d4cdc70c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=035a080de9d54a56af29cc56b33a1ddc, role_id=04a120a50aa648fea07c999a73a1c22c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-2db9364b-676f-4ce6-af38-b4b879db799a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:03.638328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:03.639209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:03.640350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:03.641006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020bc5a-5780-4ecc-83f6-555d4cdc70c6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.644741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 227/456] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:03.645684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9364b-676f-4ce6-af38-b4b879db799a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.735611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 230/457] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:08:03 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/035a080de9d54a56af29cc56b33a1ddc/roles/04a120a50aa648fea07c999a73a1c22c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:08:03.736401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020bc5a-5780-4ecc-83f6-555d4cdc70c6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.760534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9364b-676f-4ce6-af38-b4b879db799a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.800578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.801470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.801932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.802414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.810354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.812385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.819314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:03.825722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:03.826830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 228/458] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:03.827754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.853088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020bc5a-5780-4ecc-83f6-555d4cdc70c6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.884983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:03.892451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.893083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.893362 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.893639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.894293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:03.894590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.895987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:03.909909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 231/459] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:03.913589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:03.947802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92a750-0afd-461e-bb5f-3672240a4575 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:03.983063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:03.983656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:03.983934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:03.984205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:03.984947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:03.985251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:03.986742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.009807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 229/460] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:04.009807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.030124 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2062b83-472f-4a08-8ac8-096877af1f56 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.066663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.067259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.067534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.067797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.068580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:04.068895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.070322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.073177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:04.087036 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 232/461] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:08:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:04.088031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.113722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b152f8a-b90c-491d-bacf-774327a580a1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.121228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:04.147431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.148038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.148347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.148627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.149881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:04.150190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.151641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.163314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 230/462] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:04.164116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.206393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22b38d-0b74-46bd-a88b-ce048080e8eb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.243627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.244478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.244917 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.245339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.252683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.254116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:04.266698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:04.267656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 233/463] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:04.268652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.275572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2180233-734a-4f60-89d9-4afc43a546c7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.310568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f14b2f-8fbd-4bdb-8a02-03a9fbe1c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.311169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f14b2f-8fbd-4bdb-8a02-03a9fbe1c558 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.311446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f14b2f-8fbd-4bdb-8a02-03a9fbe1c558 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.311715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f14b2f-8fbd-4bdb-8a02-03a9fbe1c558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.332639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:04.347370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:04.377857 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797b85d-cf77-4391-8e0c-5e327895f0e0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.381818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f14b2f-8fbd-4bdb-8a02-03a9fbe1c558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.383843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f14b2f-8fbd-4bdb-8a02-03a9fbe1c558 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.414025 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.414680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.414961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.415236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.426337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.427780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:04.443012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:04.443960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:04.445269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:04.445269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:04.445269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:04.445269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 234/464] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:04.445269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.493054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 231/465] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:08:04.493054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f14b2f-8fbd-4bdb-8a02-03a9fbe1c558 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.554253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:04.557327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a0a4-90fc-4faa-b67a-7bacc5abef32 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.567696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:04.592196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.592402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f14b2f-8fbd-4bdb-8a02-03a9fbe1c558 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.593067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.593339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.593607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.594510 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] RBAC: Adding query filter params (role.id=04a120a50aa648fea07c999a73a1c22c, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=5c5b4b17788944f8a0991cb62b2a7205) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:04.595707 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.597178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.603728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 235/466] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/role_assignments?user.id=5c5b4b17788944f8a0991cb62b2a7205&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=04a120a50aa648fea07c999a73a1c22c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:04.604630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.643594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.644650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.645122 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.645559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.646654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:04.647184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.648898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.670509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 232/467] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:04.671792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.738812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c658f0-805f-4c29-acbc-1f026d6449ad None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.776931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.777225 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.777620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.777976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.778696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:04.779128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.780561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.803665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 236/468] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:04.803665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.821813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525ca60-0ac6-4215-94e3-c4c198b16a45 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.847260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:04.849904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:04.861600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.862357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.862732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.863102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:04.874400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:04.876465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:04.894503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:04.895693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:04.899654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:04.899654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:04.899654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:04.900343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 233/469] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:04.901295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:04.927362 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91512-42ac-4f12-b983-9d95c33dac23 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:04.966370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd504544-c73e-4e3a-b395-9f98fceb7789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:04.967250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504544-c73e-4e3a-b395-9f98fceb7789 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:04.967977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504544-c73e-4e3a-b395-9f98fceb7789 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:04.968426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504544-c73e-4e3a-b395-9f98fceb7789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.030034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff02e4-3796-4090-8384-1218b7062b0a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.046502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd504544-c73e-4e3a-b395-9f98fceb7789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.048017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd504544-c73e-4e3a-b395-9f98fceb7789 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.076113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:05.077345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:05.078073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:05.078969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.087851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:05.088748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.090393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:05.105903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:05.106885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:08:05.107777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:05.114507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 234/470] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:05.114507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:05.162052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:05.172825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 237/471] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:08:05.173667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd504544-c73e-4e3a-b395-9f98fceb7789 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:05.247829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b985bdb-9e41-4cdb-8fbb-3f2a94dff51b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.291579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:05.292367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:05.292653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:05.292940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.293868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:05.294184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.295806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.297032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd504544-c73e-4e3a-b395-9f98fceb7789 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.314632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 235/472] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:05.314632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:05.348988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:05.350671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:05.351095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:05.351524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.352993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:05.353443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.355008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.357593 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:05.379876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 238/473] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:05.379876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:05.441331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:05.451852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf9dbc-2c84-426a-901f-675a971093a0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.491527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:05.492292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:05.492694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:05.493391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.504244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.505922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.509087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e368ca-1424-47b8-93f2-6b2b161bfe6c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:05.521856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:05.522822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:05.523673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:05.523673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:05.523673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:05.528407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 236/474] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:05.529492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:05.558462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:05.559142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:05.559423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:05.559694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.570134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.573297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:05.588994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:05.589871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:05.590739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:05.590739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:05.590739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:05.590739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:05.590739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:05.590739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:05.590739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:05.590739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:05.593858 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 239/475] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:05.595232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:05.643641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f701-181c-463e-bed8-6b0cb0aebef3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.669939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:05.679586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af54531-5f42-405c-95ca-73f8f12002b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:05.683510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af54531-5f42-405c-95ca-73f8f12002b5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:05.683510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af54531-5f42-405c-95ca-73f8f12002b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:05.683510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af54531-5f42-405c-95ca-73f8f12002b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.700447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:05.737160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c98f2-8645-4442-b12f-b3657e9acfc4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.756016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af54531-5f42-405c-95ca-73f8f12002b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.757632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af54531-5f42-405c-95ca-73f8f12002b5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.786035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:05.787660 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:05.788214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:05.788724 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.789877 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:05.790812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.793014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.811876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 240/476] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:05.813038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:05.877420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 237/477] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:08:05.881894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af54531-5f42-405c-95ca-73f8f12002b5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:05.933303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:05.942954 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb1a01-349c-4034-950b-2858229d7c8f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.957845 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:05.978634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:05.979234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:05.979510 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:05.979802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:05.980524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:05.980830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:05.982254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:05.989675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af54531-5f42-405c-95ca-73f8f12002b5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:05.993927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 241/478] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:05.994712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:06.028148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84083ba-7429-4845-b9ad-3d070e70cf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:06.029140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84083ba-7429-4845-b9ad-3d070e70cf82 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.029578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84083ba-7429-4845-b9ad-3d070e70cf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.029855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84083ba-7429-4845-b9ad-3d070e70cf82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.036650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84083ba-7429-4845-b9ad-3d070e70cf82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:06.038209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84083ba-7429-4845-b9ad-3d070e70cf82 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d84083ba-7429-4845-b9ad-3d070e70cf82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:06.050429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:06.051265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:06.052113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 238/479] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:06.052625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753a401-debe-4b18-b3f5-a4a81d0a097d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.053319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753a401-debe-4b18-b3f5-a4a81d0a097d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.053319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753a401-debe-4b18-b3f5-a4a81d0a097d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.058990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 239/480] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:06.059884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753a401-debe-4b18-b3f5-a4a81d0a097d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:06.110633 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf60a5-90fd-47a8-b32b-fc486d98f899 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:06.151622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:06.152931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.153468 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.153942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.166541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:06.167648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753a401-debe-4b18-b3f5-a4a81d0a097d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:06.169031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:06.178453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:06.185108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:06.186074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:06.186948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:06.186948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:06.186948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:06.193393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 242/481] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:06.194456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:06.220441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138823e8-a780-4189-9a1a-eb34c5d0ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:06.221360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138823e8-a780-4189-9a1a-eb34c5d0ec40 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.221839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138823e8-a780-4189-9a1a-eb34c5d0ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.222333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138823e8-a780-4189-9a1a-eb34c5d0ec40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.223289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138823e8-a780-4189-9a1a-eb34c5d0ec40 None admin] RBAC: Adding query filter params (role.id=04a120a50aa648fea07c999a73a1c22c, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=32fa3c94cad04287aedea87400a15591) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:06.223893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138823e8-a780-4189-9a1a-eb34c5d0ec40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:06.226592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138823e8-a780-4189-9a1a-eb34c5d0ec40 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:06.232315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:06.243852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 240/482] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:06 2022] GET /identity/v3/role_assignments?user.id=32fa3c94cad04287aedea87400a15591&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=04a120a50aa648fea07c999a73a1c22c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:06.243852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02c36-923e-42db-ae90-ebc4e104eaff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.243852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02c36-923e-42db-ae90-ebc4e104eaff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.243852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02c36-923e-42db-ae90-ebc4e104eaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.265321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-36b02c36-923e-42db-ae90-ebc4e104eaff None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:06.266745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-36b02c36-923e-42db-ae90-ebc4e104eaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:06.411169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdfae3-730f-4b13-8ee9-38a9c8023f77 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:06.417876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 241/483] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:06.419296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b02c36-923e-42db-ae90-ebc4e104eaff None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:06.490029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:06.491368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.492164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.493723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.494816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:06.496956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:06.500330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:06.542404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 243/484] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:06.549798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:06.621538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b02c36-923e-42db-ae90-ebc4e104eaff None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:06.642867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:06.668743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:06.669841 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.670315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.670794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.671617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:06.672158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:06.673741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:06.686116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 242/485] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:06.691169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:06.740328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:06.757675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb812a-9af4-4513-ab39-df05580d7a85 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:06.799312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:06.799912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.800190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.800487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.801213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] RBAC: Adding query filter params (role.id=04a120a50aa648fea07c999a73a1c22c, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=6ee2800cf7f744a8973d5c5ca3907af1) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:06.801519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:06.803060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:06.809458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 244/486] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:06 2022] GET /identity/v3/role_assignments?user.id=6ee2800cf7f744a8973d5c5ca3907af1&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=04a120a50aa648fea07c999a73a1c22c => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:06.810306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:06.815729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2b836-0ad3-4fa0-9557-2dc741647f6b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:06.903368 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:06.904027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:06.904326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:06.904602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:06.918157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:06.919640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:06.938569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:06.939482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:06.940372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 243/487] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:06.946376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:06.971747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee1669-200e-472d-b762-63cce7f067bd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:06.977059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:07.028725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c088ee7-60e8-4e89-a877-03fa48b7d6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:07.029263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088ee7-60e8-4e89-a877-03fa48b7d6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.029545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088ee7-60e8-4e89-a877-03fa48b7d6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.029820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088ee7-60e8-4e89-a877-03fa48b7d6f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.043316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c088ee7-60e8-4e89-a877-03fa48b7d6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7c088ee7-60e8-4e89-a877-03fa48b7d6f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:07.048628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:07.049522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:07.050330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 245/488] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:07.050989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eb54b-284b-45ed-b194-c631f9c99850 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.051566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eb54b-284b-45ed-b194-c631f9c99850 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.051566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073eb54b-284b-45ed-b194-c631f9c99850 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.053936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 246/489] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:07.055441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073eb54b-284b-45ed-b194-c631f9c99850 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:07.067252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:07.074453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5d4d1-749c-4baa-9e9e-faa5b57ea791 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:07.132729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f3542-3a6c-4797-8aac-16e998443bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:07.133890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f3542-3a6c-4797-8aac-16e998443bcd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.134166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f3542-3a6c-4797-8aac-16e998443bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.134705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f3542-3a6c-4797-8aac-16e998443bcd None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.135346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f3542-3a6c-4797-8aac-16e998443bcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:07.135657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f3542-3a6c-4797-8aac-16e998443bcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:07.137339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f3542-3a6c-4797-8aac-16e998443bcd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:07.150570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 244/490] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:07.151817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7431e06-b70b-446e-9f90-9d381d19709b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.152102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7431e06-b70b-446e-9f90-9d381d19709b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.157146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7431e06-b70b-446e-9f90-9d381d19709b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.175965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a7431e06-b70b-446e-9f90-9d381d19709b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:07.177174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a7431e06-b70b-446e-9f90-9d381d19709b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:07.187792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073eb54b-284b-45ed-b194-c631f9c99850 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:07.233339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:07.234756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.235241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.235949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.236966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:07.237455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:07.239616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:07.248448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 245/491] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:07.249586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7431e06-b70b-446e-9f90-9d381d19709b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:07.250895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 247/492] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:08:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:07.256726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:07.280744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:07.340681 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:07.379627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc90db9-84ca-43db-b434-4f4d81eff6fa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:07.438549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:07.439569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.440053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.441478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.465355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7431e06-b70b-446e-9f90-9d381d19709b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:07.468606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:07.470057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:07.493476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:07.494609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:07.495487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:07.495487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:07.495487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:07.496102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 248/493] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:07.501582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:07.521195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:07.521801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.522078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.522345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.523481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] RBAC: Adding query filter params (role.id=04a120a50aa648fea07c999a73a1c22c, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=65d4ca1062fe496dbe16d6cd1bcac1a8) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:07.523825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:07.525276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:07.531879 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 246/494] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:07 2022] GET /identity/v3/role_assignments?user.id=65d4ca1062fe496dbe16d6cd1bcac1a8&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=04a120a50aa648fea07c999a73a1c22c => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:07.644174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ec1b5-f442-47f5-b7c4-167a0c026bf7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:07.676581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:07.691698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:07.694355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:07.695241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.704326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.704326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.704326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:07.704326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:07.704326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:07.713974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 249/495] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:07.724060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:07.820033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:07.845129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6283eb9-05b0-4758-be4c-6379d7924b2f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:07.857319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaa40a-6b7e-48cd-bc9a-62fdffe1b557 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:07.897339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fadc4-e4c3-476f-a6ed-4c1ad2eb94c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:07.897958 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fadc4-e4c3-476f-a6ed-4c1ad2eb94c6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.898241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fadc4-e4c3-476f-a6ed-4c1ad2eb94c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.898500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fadc4-e4c3-476f-a6ed-4c1ad2eb94c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.902204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:07.903434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:07.905379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:07.905914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:07.914632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:07.916650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:07.924883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fadc4-e4c3-476f-a6ed-4c1ad2eb94c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-361fadc4-e4c3-476f-a6ed-4c1ad2eb94c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:07.930194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:07.931211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:07.932048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 247/496] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:07.939012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:07.939930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:08:07.940960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:07.943745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 250/497] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:07.948386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fadc4-e4c3-476f-a6ed-4c1ad2eb94c6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:07.950236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:07.999820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:08.104313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7f212-b5a1-4d29-b73f-655e86f9b475 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:08.147283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fadc4-e4c3-476f-a6ed-4c1ad2eb94c6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:08.167835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:08.168483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:08.168761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:08.169028 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:08.170909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:08.171221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:08.172663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:08.210919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 251/498] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Oct 24 13:08:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:08.217803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:08.225039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:08.225671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:08.225928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:08.229076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:08.229764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:08.236518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:08.238202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:08.247381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:08.257715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 248/499] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:08.263163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:08.393144 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:08.396762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648f574-6c29-4ea4-b0e9-85eee81df0c6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:08.421264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f5ae9-a6da-4c76-bc8e-c2bf79062a25 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:08.443869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc438ed-2700-4383-972e-807a35e60261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:08.444571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc438ed-2700-4383-972e-807a35e60261 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:08.444796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc438ed-2700-4383-972e-807a35e60261 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:08.445068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc438ed-2700-4383-972e-807a35e60261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:08.484599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:08.485223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:08.485497 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:08.485766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:08.500071 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:08.501549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:08.521714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:08.523188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:08.524832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:08.524832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:08.524832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:08.524832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 249/500] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:08.528813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:08.587486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc438ed-2700-4383-972e-807a35e60261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:08.591697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc438ed-2700-4383-972e-807a35e60261 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:08.620624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:08.729144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f495c-8cd4-42fa-a3c2-cb0ea7db9135 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:08.780830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:08.788520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 252/501] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:08:08.792049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc438ed-2700-4383-972e-807a35e60261 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:08.811641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:08.812253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:08.812596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:08.812894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:08.813536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:08.813836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:08.815278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:08.829386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 250/502] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:08.834386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:08.935698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc438ed-2700-4383-972e-807a35e60261 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:08.949003 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072a573-bfff-40f3-bbf2-7d8296c5d361 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:08.960370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:08.975778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8f2bd-9953-45bc-b159-f10fde4df7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:08.976448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8f2bd-9953-45bc-b159-f10fde4df7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:08.976727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8f2bd-9953-45bc-b159-f10fde4df7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:08.977061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8f2bd-9953-45bc-b159-f10fde4df7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:08.986735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:08.987573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:08.987997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:08.988140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8f2bd-9953-45bc-b159-f10fde4df7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:08.988569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:08.989612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8f2bd-9953-45bc-b159-f10fde4df7f9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:08.998191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:08.999956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7df8f2bd-9953-45bc-b159-f10fde4df7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:09.006758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:09.007883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:09.009422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:09.009422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:09.009422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:09.012830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 253/503] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:09.014050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa3494-0bd1-4848-8663-071880730a48 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.014331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa3494-0bd1-4848-8663-071880730a48 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.014605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa3494-0bd1-4848-8663-071880730a48 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:09.015575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:09.020561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:09.021597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:09.022141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 254/504] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:09.022141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa3494-0bd1-4848-8663-071880730a48 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.024111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 251/505] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Oct 24 13:08:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:09.030006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.081112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:09.135233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa3494-0bd1-4848-8663-071880730a48 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.143019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e3497-fbbe-4935-abf8-98837927fa2b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.175892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd309c-94e4-45fa-94db-f279569ec826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.176583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd309c-94e4-45fa-94db-f279569ec826 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.177193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd309c-94e4-45fa-94db-f279569ec826 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.177193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd309c-94e4-45fa-94db-f279569ec826 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.177821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd309c-94e4-45fa-94db-f279569ec826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:09.178126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd309c-94e4-45fa-94db-f279569ec826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.179536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd309c-94e4-45fa-94db-f279569ec826 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.186661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.187634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.188230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.188728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.189671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:09.190317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.192070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.198267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 255/506] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:09.199573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807359cf-140e-4650-8774-5a04ca2ed40f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.199866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807359cf-140e-4650-8774-5a04ca2ed40f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.200131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807359cf-140e-4650-8774-5a04ca2ed40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.213461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 252/507] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:09.219254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.220802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-807359cf-140e-4650-8774-5a04ca2ed40f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:09.221978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-807359cf-140e-4650-8774-5a04ca2ed40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:09.255700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:09.293781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 256/508] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:09.294905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807359cf-140e-4650-8774-5a04ca2ed40f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.334283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ff565-c19b-47be-abaf-317fb30376a5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.345044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:09.369900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c4616-baa0-4739-bf19-17edbeea2c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.370547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c4616-baa0-4739-bf19-17edbeea2c26 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.370828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c4616-baa0-4739-bf19-17edbeea2c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.371100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c4616-baa0-4739-bf19-17edbeea2c26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.378024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c4616-baa0-4739-bf19-17edbeea2c26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.379475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c4616-baa0-4739-bf19-17edbeea2c26 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-eb6c4616-baa0-4739-bf19-17edbeea2c26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:09.392813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:09.393728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:09.394628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:09.394628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:09.394628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:09.394628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:09.394628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:09.394628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:09.394628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:09.394628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:09.398771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 253/509] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:09.400677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7382e2d-17f6-429d-98ce-cf2d5a0428d9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.400965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7382e2d-17f6-429d-98ce-cf2d5a0428d9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.401238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7382e2d-17f6-429d-98ce-cf2d5a0428d9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.402745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 254/510] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:09.403482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7382e2d-17f6-429d-98ce-cf2d5a0428d9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.420432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807359cf-140e-4650-8774-5a04ca2ed40f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.465098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49eaa4e-ed7f-44d5-a29c-a0a2ab2b99a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.466572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eaa4e-ed7f-44d5-a29c-a0a2ab2b99a8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.467620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eaa4e-ed7f-44d5-a29c-a0a2ab2b99a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.468069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eaa4e-ed7f-44d5-a29c-a0a2ab2b99a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.481672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49eaa4e-ed7f-44d5-a29c-a0a2ab2b99a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.483299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49eaa4e-ed7f-44d5-a29c-a0a2ab2b99a8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.498777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e49eaa4e-ed7f-44d5-a29c-a0a2ab2b99a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:09.501946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:09.503071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:09.504156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:09.504156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:09.504156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:09.506503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 257/511] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:09.507872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbd2fd-b75b-4b37-8f2e-6dada6e9daeb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.508848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbd2fd-b75b-4b37-8f2e-6dada6e9daeb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.509253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbd2fd-b75b-4b37-8f2e-6dada6e9daeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.520157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7382e2d-17f6-429d-98ce-cf2d5a0428d9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.532012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bbbbd2fd-b75b-4b37-8f2e-6dada6e9daeb None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:09.533673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bbbbd2fd-b75b-4b37-8f2e-6dada6e9daeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:09.565824 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.566956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.567781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.568356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.569306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:09.569919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.571920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.585546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 255/512] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:09.586285 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.611849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 258/513] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:09.613107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbd2fd-b75b-4b37-8f2e-6dada6e9daeb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.641104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:09.719166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbac4c3-47c6-4407-97ef-8d9e96db773b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.721562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbd2fd-b75b-4b37-8f2e-6dada6e9daeb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.758911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.759571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.759920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.760278 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.760941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.762605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.764240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.765316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.765781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.766234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.767430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:09.768150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.769760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:09.774972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:09.775989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:09.777252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 259/514] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:09.778862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.787797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 256/515] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:09.788918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.801588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:09.895522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148ff68-f803-4f09-a1fd-a3ad438fb66e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.900304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:09.912693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8f715-6440-4865-b0f1-abeea396bae5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:09.939224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.939844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.940211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.940428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.941052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:09.941352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.942781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.947621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 257/516] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:09.948480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:09.958747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:09.959614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:09.959898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:09.960178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:09.960993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] RBAC: Adding query filter params (role.id=04a120a50aa648fea07c999a73a1c22c, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=035a080de9d54a56af29cc56b33a1ddc) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:09.961339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:09.962888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:09.969239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 260/517] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/role_assignments?user.id=035a080de9d54a56af29cc56b33a1ddc&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=04a120a50aa648fea07c999a73a1c22c => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:09.970019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:10.052053 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:10.091395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a8464-18e2-4db8-825d-0bdde948b744 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:10.094638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f87c6-2800-413e-94d1-f2dee0cfa006 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:10.130046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:10.130693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.130963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.131501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.136868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5efc1d-32dd-4454-aa7e-993b64c3ccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:10.137584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5efc1d-32dd-4454-aa7e-993b64c3ccdc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.137925 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5efc1d-32dd-4454-aa7e-993b64c3ccdc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.138261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5efc1d-32dd-4454-aa7e-993b64c3ccdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.142509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:10.143955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:10.147645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5efc1d-32dd-4454-aa7e-993b64c3ccdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:10.149152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5efc1d-32dd-4454-aa7e-993b64c3ccdc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:10.159823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:10.160893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:10.161926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:10.161926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:10.161926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:10.161926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 261/518] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:10.162822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9a5efc1d-32dd-4454-aa7e-993b64c3ccdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:10.166827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:10.167712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:10.168695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:10.168695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:10.168695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:10.168695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:10.168695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:10.168695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:10.168695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:10.168695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:10.169153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:10.169245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 258/519] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:10.169444 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231a75-8549-4541-8fc3-538d54a18e7d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.170039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231a75-8549-4541-8fc3-538d54a18e7d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.170039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231a75-8549-4541-8fc3-538d54a18e7d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.172211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 259/520] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:10.172950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231a75-8549-4541-8fc3-538d54a18e7d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:10.279503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231a75-8549-4541-8fc3-538d54a18e7d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:10.280951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea907ff2-d995-4586-b7b0-a4cd22ab3db1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:10.310015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:10.315219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb4bbd-8e9d-4da0-86b7-6799eb8876b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:10.315812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4bbd-8e9d-4da0-86b7-6799eb8876b3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.316086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4bbd-8e9d-4da0-86b7-6799eb8876b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.316390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4bbd-8e9d-4da0-86b7-6799eb8876b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.316686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:10.317077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4bbd-8e9d-4da0-86b7-6799eb8876b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:10.317280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.317388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4bbd-8e9d-4da0-86b7-6799eb8876b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:10.317559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.317901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.318459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:10.318757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:10.318847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4bbd-8e9d-4da0-86b7-6799eb8876b3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:10.320100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:10.331809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 260/521] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:10.333353 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b892-de7d-4149-ac7b-1c8e0b98c42d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.333771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b892-de7d-4149-ac7b-1c8e0b98c42d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.334170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b892-de7d-4149-ac7b-1c8e0b98c42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.336992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 262/522] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Oct 24 13:08:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:10.337988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:10.358181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-40c7b892-de7d-4149-ac7b-1c8e0b98c42d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:10.359731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-40c7b892-de7d-4149-ac7b-1c8e0b98c42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:10.419916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:10.447133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 261/523] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:10.448407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7b892-de7d-4149-ac7b-1c8e0b98c42d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:10.480798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e97701-03aa-4156-ae70-bb17e3268055 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:10.537301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:10.537908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.538191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.538460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.539169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:10.539480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:10.541012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:10.556803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 263/524] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:10.563347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:10.619540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7b892-de7d-4149-ac7b-1c8e0b98c42d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:10.663827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:10.690447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a25136-c4b6-4208-9622-bd74896eff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:10.691050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25136-c4b6-4208-9622-bd74896eff0c None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.691322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25136-c4b6-4208-9622-bd74896eff0c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.691587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25136-c4b6-4208-9622-bd74896eff0c None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.692301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a25136-c4b6-4208-9622-bd74896eff0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:10.694638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a25136-c4b6-4208-9622-bd74896eff0c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:10.725062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b14b-7ec2-4391-8201-f9ff9ca90b71 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:10.752806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:10.760838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360d7ef-5b51-445f-b132-023cf065d5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:10.761490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360d7ef-5b51-445f-b132-023cf065d5ca None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.761764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360d7ef-5b51-445f-b132-023cf065d5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.762031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360d7ef-5b51-445f-b132-023cf065d5ca None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/5c5b4b17788944f8a0991cb62b2a7205/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.764071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360d7ef-5b51-445f-b132-023cf065d5ca None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=5c5b4b17788944f8a0991cb62b2a7205, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:10.765618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360d7ef-5b51-445f-b132-023cf065d5ca None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:10.792677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 262/525] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:08:10 2022] POST /identity/v3/users => generated 254 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:10.793549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a25136-c4b6-4208-9622-bd74896eff0c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:10.845179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 264/526] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:08:10 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/5c5b4b17788944f8a0991cb62b2a7205/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:08:10.928541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a25136-c4b6-4208-9622-bd74896eff0c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:10.969326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:10.970079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:10.970360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:10.970628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] PATH_INFO: `/v3/services/network` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:10.971334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:10.973009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:10.985296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:10.986693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:10.987950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 263/527] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:10 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:10.992309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360d7ef-5b51-445f-b132-023cf065d5ca None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:11.112331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360d7ef-5b51-445f-b132-023cf065d5ca None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:11.150181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a247f4c-0fb8-4dad-96c5-b9648b47ef69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:11.150789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a247f4c-0fb8-4dad-96c5-b9648b47ef69 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:11.151069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a247f4c-0fb8-4dad-96c5-b9648b47ef69 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:11.151345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a247f4c-0fb8-4dad-96c5-b9648b47ef69 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:11.151977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a247f4c-0fb8-4dad-96c5-b9648b47ef69 None admin] RBAC: Adding query filter params (name=network) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:11.152293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a247f4c-0fb8-4dad-96c5-b9648b47ef69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:11.153669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a247f4c-0fb8-4dad-96c5-b9648b47ef69 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:11.158508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 265/528] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:11.163773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:11.240148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:11.308242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fea01-d995-405d-92cc-f02966de8cbf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:11.343289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8132a63-a9c2-4be6-b2d3-19d59347ec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:11.343884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8132a63-a9c2-4be6-b2d3-19d59347ec17 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:11.344159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8132a63-a9c2-4be6-b2d3-19d59347ec17 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:11.344452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8132a63-a9c2-4be6-b2d3-19d59347ec17 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:11.345075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8132a63-a9c2-4be6-b2d3-19d59347ec17 None admin] RBAC: Adding query filter params (type=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:11.345371 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8132a63-a9c2-4be6-b2d3-19d59347ec17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:11.346743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8132a63-a9c2-4be6-b2d3-19d59347ec17 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:11.353331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 264/529] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:12.466969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d50b9-7c09-4a28-b9f1-efc7f6720633 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:12.467790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d50b9-7c09-4a28-b9f1-efc7f6720633 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:12.467790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d50b9-7c09-4a28-b9f1-efc7f6720633 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:12.469410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 266/530] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:12.474592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86327a94-1263-4751-84d3-c00ed2ac95db None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:12.475022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86327a94-1263-4751-84d3-c00ed2ac95db None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:12.475322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86327a94-1263-4751-84d3-c00ed2ac95db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:12.500241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-86327a94-1263-4751-84d3-c00ed2ac95db None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:12.501586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-86327a94-1263-4751-84d3-c00ed2ac95db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:12.590476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 265/531] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:12.890060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d50b9-7c09-4a28-b9f1-efc7f6720633 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:12.939096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee007a5-a797-4baf-a7a9-922268b55ecc None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:12.939450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee007a5-a797-4baf-a7a9-922268b55ecc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:12.939658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee007a5-a797-4baf-a7a9-922268b55ecc None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:12.941256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 266/532] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:12.946528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5006-87c0-4012-ba4d-7b5768b4af56 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:12.946809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5006-87c0-4012-ba4d-7b5768b4af56 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:12.947084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca5006-87c0-4012-ba4d-7b5768b4af56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:12.952149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:12.985253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c8ca5006-87c0-4012-ba4d-7b5768b4af56 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:12.986438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c8ca5006-87c0-4012-ba4d-7b5768b4af56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:13.002386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d50b9-7c09-4a28-b9f1-efc7f6720633 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:13.039500 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3879344-9c28-4d6b-883f-c886615162cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:13.040133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879344-9c28-4d6b-883f-c886615162cc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.040567 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879344-9c28-4d6b-883f-c886615162cc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.040853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3879344-9c28-4d6b-883f-c886615162cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.051856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3879344-9c28-4d6b-883f-c886615162cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b3879344-9c28-4d6b-883f-c886615162cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:13.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:13.058542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:13.059421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:13.060078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:13.060154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 267/533] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:13.066744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3879344-9c28-4d6b-883f-c886615162cc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:13.138437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 267/534] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:13.185159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3879344-9c28-4d6b-883f-c886615162cc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:13.216643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:13.234761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:13.235400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.235684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.235961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.236652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:13.236956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:13.238314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:13.247429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 268/535] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:13.252978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca5006-87c0-4012-ba4d-7b5768b4af56 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:13.404657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca5006-87c0-4012-ba4d-7b5768b4af56 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:13.457939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:13.457939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.457939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.458284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.469212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:13.470714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:13.491558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:13.493567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:13.494478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:13.494478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:13.494478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:13.496828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 268/536] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:13.502644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:13.517587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:13.537193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:13.648660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9708e0-598b-4797-afa6-b04113efdfeb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:13.652143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec898380-29b1-40e0-8fd3-4561b8d22b71 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:13.703911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf5d2-a6fe-4370-ad8a-0d189ac5ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:13.704540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf5d2-a6fe-4370-ad8a-0d189ac5ad14 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.704817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf5d2-a6fe-4370-ad8a-0d189ac5ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.705084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf5d2-a6fe-4370-ad8a-0d189ac5ad14 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.705720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caf5d2-a6fe-4370-ad8a-0d189ac5ad14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:13.706018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caf5d2-a6fe-4370-ad8a-0d189ac5ad14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:13.707393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caf5d2-a6fe-4370-ad8a-0d189ac5ad14 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:13.719279 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edade2-f22f-4c3b-8f2b-e9b47752a7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:13.719820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edade2-f22f-4c3b-8f2b-e9b47752a7e6 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.720096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edade2-f22f-4c3b-8f2b-e9b47752a7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.720396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edade2-f22f-4c3b-8f2b-e9b47752a7e6 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.721077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edade2-f22f-4c3b-8f2b-e9b47752a7e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:13.722495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edade2-f22f-4c3b-8f2b-e9b47752a7e6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:13.736043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 269/537] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:13.737473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd3421-d2fd-4cb7-bc04-4169d6e600d4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.737803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd3421-d2fd-4cb7-bc04-4169d6e600d4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.738075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd3421-d2fd-4cb7-bc04-4169d6e600d4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.739641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 270/538] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:13.743238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd3421-d2fd-4cb7-bc04-4169d6e600d4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:13.781164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:13.839038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 269/539] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:08:13 2022] POST /identity/v3/services => generated 251 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:08:13.840439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9de81-3ad8-427f-8472-ea6715c90b56 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.840727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9de81-3ad8-427f-8472-ea6715c90b56 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.840997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9de81-3ad8-427f-8472-ea6715c90b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.864687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-68f9de81-3ad8-427f-8472-ea6715c90b56 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:13.865951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-68f9de81-3ad8-427f-8472-ea6715c90b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:13.881672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd3421-d2fd-4cb7-bc04-4169d6e600d4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:13.933619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:13.934972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.935462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.935991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.946587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:13.946781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 270/540] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:13.948006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a208b-6c57-418d-8aa8-88ff5f5346b9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.948333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a208b-6c57-418d-8aa8-88ff5f5346b9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.948658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a208b-6c57-418d-8aa8-88ff5f5346b9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.949439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:13.950221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 271/541] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:13.960322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43dc33-b314-418d-ba0a-7db753ee72d0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:13.960322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43dc33-b314-418d-ba0a-7db753ee72d0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:13.960322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43dc33-b314-418d-ba0a-7db753ee72d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:13.980848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:13.981946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:08:13.982958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:13.985346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 271/542] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:13.990767 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:13.994079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2c43dc33-b314-418d-ba0a-7db753ee72d0 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:13.999146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2c43dc33-b314-418d-ba0a-7db753ee72d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:14.002836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:14.112812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 272/543] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:14.122507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db94573-c211-4dca-8f79-bacd10bcba34 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.122800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db94573-c211-4dca-8f79-bacd10bcba34 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.123078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db94573-c211-4dca-8f79-bacd10bcba34 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.124720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 273/544] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:14.130325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f2dd-f60b-4aee-b3aa-670d7349f77e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.130614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f2dd-f60b-4aee-b3aa-670d7349f77e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.130890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f2dd-f60b-4aee-b3aa-670d7349f77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.139804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:14.150394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edba80-9031-4736-a797-52d2bafac717 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:14.150801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9199f2dd-f60b-4aee-b3aa-670d7349f77e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:14.152098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9199f2dd-f60b-4aee-b3aa-670d7349f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:14.190263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:14.191008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.191289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.191562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.192572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:14.192887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:14.194986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:14.218432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 272/545] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Oct 24 13:08:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:14.224424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.244879 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 274/546] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:14 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:08:14.252290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199f2dd-f60b-4aee-b3aa-670d7349f77e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.332250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911928ed-3864-449e-a62b-b20acccdd7be None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:14.370609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:14.371238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.371513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.371782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.391803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:14.393303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:14.399580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:14.405180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:14.411995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:14.419314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:14.420248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:14.420248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:14.420248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:14.420248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 273/547] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:14.420248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.442966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199f2dd-f60b-4aee-b3aa-670d7349f77e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:14.481320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207479-cabd-4eb3-8658-e0e0526517e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:14.482198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207479-cabd-4eb3-8658-e0e0526517e9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.482647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207479-cabd-4eb3-8658-e0e0526517e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.483011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e207479-cabd-4eb3-8658-e0e0526517e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.494318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e207479-cabd-4eb3-8658-e0e0526517e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-9e207479-cabd-4eb3-8658-e0e0526517e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:14.499658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:14.500631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:14.501500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 275/548] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:14.502206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207479-cabd-4eb3-8658-e0e0526517e9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.543810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3e69c-377f-4189-a8a9-0a08a0875792 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:14.583564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd54fd-4739-4938-87ee-c2de5c808396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:14.584296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd54fd-4739-4938-87ee-c2de5c808396 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.584492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd54fd-4739-4938-87ee-c2de5c808396 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.584964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd54fd-4739-4938-87ee-c2de5c808396 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.598421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd54fd-4739-4938-87ee-c2de5c808396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-89cd54fd-4739-4938-87ee-c2de5c808396 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:14.604828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:14.605769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:14.606624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:14.608139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 274/549] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:14.608244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd54fd-4739-4938-87ee-c2de5c808396 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.623889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e207479-cabd-4eb3-8658-e0e0526517e9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:14.651465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:14.660245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:14.669189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.669189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.669189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.669189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:14.669189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:14.669189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:14.672667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:14.682164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 276/550] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:14.682164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.713998 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd54fd-4739-4938-87ee-c2de5c808396 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:14.749226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:14.749885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.750108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.750385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.751044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:14.751348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:14.752755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:14.761893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 275/551] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:14.762694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.786523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbcbc5-2ce0-4e5e-95a7-aad4ce69cc7e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:14.821082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ff51-c1fe-4154-bd80-692605374dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:14.821686 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ff51-c1fe-4154-bd80-692605374dbd None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.821967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ff51-c1fe-4154-bd80-692605374dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.822239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44ff51-c1fe-4154-bd80-692605374dbd None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.822878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44ff51-c1fe-4154-bd80-692605374dbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:14.824253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44ff51-c1fe-4154-bd80-692605374dbd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:14.865054 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9ae5d-b703-47c6-a705-c1e3535c11b9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:14.888923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:14.900131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:14.900768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:14.901106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:14.901381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:14.902040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:14.902342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:14.903700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:14.912602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 276/552] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:08:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:14.913253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.920345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 277/553] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:08:14 2022] POST /identity/v3/services => generated 240 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:14.922075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ff51-c1fe-4154-bd80-692605374dbd None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:14.995534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:15.057234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbb983-6cf5-4bb2-8cd5-159848c7cbec None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.061924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44ff51-c1fe-4154-bd80-692605374dbd None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.102912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.103665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.104047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.105222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.106488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.106488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.106687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.107470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.114256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.116149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:15.119290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.121118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:15.129915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:15.131001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:15.132105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:15.132105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:15.132105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:15.132105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:15.132105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:15.132105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:15.132105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:15.132105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:15.134189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 277/554] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:15.134959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:15.146492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:15.147572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:15.149186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:15.149186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:15.149186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:15.150665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 278/555] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:15.151571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:15.186427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:15.254011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:15.258987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df252a28-77a4-40b6-ab70-6a468fe0058d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.260191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718131b-40a2-4ed6-ace8-e02435fea803 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.296749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.297489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.297814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.298154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.300132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.300760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.301042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.301343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.302166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:15.302472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.303887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:15.311485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.313049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:15.318327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 279/556] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:15.319455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:15.328336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:15.329538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:15.330661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:15.330661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:15.330661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:15.330661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 278/557] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:15.330661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:15.443413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:15.445881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544d2f9-d1a7-40f5-b2ad-14374e7e94e2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.449964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b774865-3908-4d51-9e14-1387baed1fba None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.482337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.482957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.483218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.483487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.484160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:15.484490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.485842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:15.488607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71641cc-8b6c-4b18-a41e-45c35ab86b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.488905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71641cc-8b6c-4b18-a41e-45c35ab86b9a None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.489355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71641cc-8b6c-4b18-a41e-45c35ab86b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.489634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71641cc-8b6c-4b18-a41e-45c35ab86b9a None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/035a080de9d54a56af29cc56b33a1ddc/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.491580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71641cc-8b6c-4b18-a41e-45c35ab86b9a None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=035a080de9d54a56af29cc56b33a1ddc, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.493592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71641cc-8b6c-4b18-a41e-45c35ab86b9a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:15.499776 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 280/558] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:15.500670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:15.554366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:15.592652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 279/559] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:08:15 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/035a080de9d54a56af29cc56b33a1ddc/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:08:15.593768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71641cc-8b6c-4b18-a41e-45c35ab86b9a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:15.651901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0697c-d868-4f3d-b6a5-bfe14ec89963 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.696224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.697105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.698139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.698609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.699698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:15.700177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.701984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:15.717627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 281/560] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:15.723581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:15.794312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71641cc-8b6c-4b18-a41e-45c35ab86b9a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.843052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.843907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.844387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.844798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.855965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.858366 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:15.873321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:15.874493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:15.875641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:15.876642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:15.879838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 280/561] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Oct 24 13:08:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:15.886496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:15.907314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32650-f65d-438e-94fb-cfc7ff28aaa2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:15.908832 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:15.964708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:15.965751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:15.966247 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:15.966734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:15.979145 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:15.982107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:16.010645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:16.011722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:08:16.012729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:16.015002 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 282/562] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Oct 24 13:08:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:16.019999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:16.062083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf655af-6bc4-4664-9d00-58c1839e3064 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:16.130072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:16.131059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.132577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.133544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.135087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:16.136059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:16.137794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:16.179114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 281/563] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Oct 24 13:08:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:16.181245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:16.214691 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25418cc9-eab6-4df3-850a-0eb2a0fcb644 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:16.287592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab5a6a-9428-495f-ac8f-dce175c42883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:16.288459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5a6a-9428-495f-ac8f-dce175c42883 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.289861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5a6a-9428-495f-ac8f-dce175c42883 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.290377 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5a6a-9428-495f-ac8f-dce175c42883 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.294156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab5a6a-9428-495f-ac8f-dce175c42883 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:16.294620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab5a6a-9428-495f-ac8f-dce175c42883 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:16.296199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab5a6a-9428-495f-ac8f-dce175c42883 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:16.303260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:16.333815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 283/564] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Oct 24 13:08:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:16.335117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903809f-883e-407a-adf1-854578c752cc None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.335402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903809f-883e-407a-adf1-854578c752cc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.335670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903809f-883e-407a-adf1-854578c752cc None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.336062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:16.346999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 284/565] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:16.348174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac2197-df5b-476b-b2af-7bcf4e888548 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.348486 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac2197-df5b-476b-b2af-7bcf4e888548 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.348816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac2197-df5b-476b-b2af-7bcf4e888548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.366500 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9626-9307-4c36-b80e-06ed516777eb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:16.367096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d6ac2197-df5b-476b-b2af-7bcf4e888548 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:16.368917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d6ac2197-df5b-476b-b2af-7bcf4e888548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:16.402009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb4be3-5e4e-46fd-9b0a-b83dbc7140e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:16.402608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb4be3-5e4e-46fd-9b0a-b83dbc7140e6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.402884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb4be3-5e4e-46fd-9b0a-b83dbc7140e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.403154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb4be3-5e4e-46fd-9b0a-b83dbc7140e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.440182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 285/566] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:16.441269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac2197-df5b-476b-b2af-7bcf4e888548 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:16.471738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb4be3-5e4e-46fd-9b0a-b83dbc7140e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:16.473242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb4be3-5e4e-46fd-9b0a-b83dbc7140e6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:16.547258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:16.563448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac2197-df5b-476b-b2af-7bcf4e888548 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:16.576391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 282/567] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:08:16.578758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb4be3-5e4e-46fd-9b0a-b83dbc7140e6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:16.607803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:16.627385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:16.628022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.628323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.628603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.642621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:16.645040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:16.668905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:16.670107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:16.671146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:16.671146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:16.671146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:16.671146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 286/568] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:16.677655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:16.693579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb4be3-5e4e-46fd-9b0a-b83dbc7140e6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:16.728996 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:16.729616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.729894 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.730163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.741091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:16.742585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:16.759033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:16.760169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:16.761359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:16.761359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:16.761359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:16.764907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 283/569] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:16.766332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:16.794702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:16.844910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89baec-6b89-4d92-9236-5d4a364aec00 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:16.872971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7387a7e-dc2d-4b64-8859-b22379864843 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:16.890133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:16.891134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.891800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.892224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.893460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:16.893913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:16.895419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:16.910959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:16.911637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:16.911980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:16.912387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:16.912788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 287/570] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:16.913429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:16.913937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:16.914904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:16.920301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:16.937892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 284/571] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:16.938849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:16.956498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:17.024352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501b789-7f4c-4548-ab94-1a56d82a8373 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.040961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:17.058619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9c222-ad04-47ee-bfc2-5fbb3bd16c6a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.060055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.061055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.061537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.062138 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.062917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.064532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:17.075431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:17.076505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:17.077852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 288/572] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Oct 24 13:08:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:17.079657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:17.107687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.108340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.108623 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.109140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.118107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.119662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:17.134320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:17.135222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 285/573] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:17.137080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:17.204204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:17.207342 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50111518-584e-4dbd-91e6-ca689fbc4ae6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.246174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.246972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.247407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.247742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.253781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571eff25-d6c1-4d0b-ad83-6497e670e8f7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.256003 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.258514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:17.272399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:17.273568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:17.274715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:17.274715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:17.274715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:17.274715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:17.274715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:17.274715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:17.274715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:17.274715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:17.275940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 289/574] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:17.275940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:17.290672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.291280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.291734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.292016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.292830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:17.293149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.295333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.306106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 286/575] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Oct 24 13:08:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:17.307260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:17.322125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:17.394502 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f543f-bbdc-4574-8033-01f8f5c1845a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.421266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebaa22-5a48-4d42-ba92-2432932e0a9f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.428750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.429343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.429638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.430461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.431135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:17.431436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.432884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.445016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 290/576] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:17.449058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:17.463725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2340c-f054-435e-b78e-a9a2d3421fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.464363 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2340c-f054-435e-b78e-a9a2d3421fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.464642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2340c-f054-435e-b78e-a9a2d3421fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.464909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2340c-f054-435e-b78e-a9a2d3421fb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.465594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2340c-f054-435e-b78e-a9a2d3421fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:17.465976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2340c-f054-435e-b78e-a9a2d3421fb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.467648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea2340c-f054-435e-b78e-a9a2d3421fb6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.470716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:17.485152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 287/577] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:17.485152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef49719-1692-47f8-9ce0-87b27a84e30c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.485152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef49719-1692-47f8-9ce0-87b27a84e30c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.485152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef49719-1692-47f8-9ce0-87b27a84e30c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.485152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 288/578] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:17.485152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef49719-1692-47f8-9ce0-87b27a84e30c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:17.560685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7249d-937f-42c3-a0e5-df4e3abe79e4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.561651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:17.594627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef49719-1692-47f8-9ce0-87b27a84e30c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.596029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.596662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.596999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.597281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.597905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:17.598212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.599575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.609653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 291/579] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Oct 24 13:08:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:17.609653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:17.636367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e89fef-57c1-4229-bbb5-854992429642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.637020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e89fef-57c1-4229-bbb5-854992429642 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.637294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e89fef-57c1-4229-bbb5-854992429642 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.637565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e89fef-57c1-4229-bbb5-854992429642 None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/038f2ef649844de297237a0c1ec652aa/roles/04a120a50aa648fea07c999a73a1c22c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.652527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e89fef-57c1-4229-bbb5-854992429642 None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=038f2ef649844de297237a0c1ec652aa, role_id=04a120a50aa648fea07c999a73a1c22c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.654051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e89fef-57c1-4229-bbb5-854992429642 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.693584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:17.708498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d4968-cbff-48eb-ba04-845ca47f4a0a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.741238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad738f-2e39-4fee-9566-10e528e4dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.741825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad738f-2e39-4fee-9566-10e528e4dfec None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.742124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad738f-2e39-4fee-9566-10e528e4dfec None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.742941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad738f-2e39-4fee-9566-10e528e4dfec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.743652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad738f-2e39-4fee-9566-10e528e4dfec None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=5c5b4b17788944f8a0991cb62b2a7205) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:17.743956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad738f-2e39-4fee-9566-10e528e4dfec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.745416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad738f-2e39-4fee-9566-10e528e4dfec None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.760381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 292/580] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:17 2022] GET /identity/v3/role_assignments?user.id=5c5b4b17788944f8a0991cb62b2a7205&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:17.760381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992c323-19d6-4d34-a6bf-3890b22e4d94 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.760381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992c323-19d6-4d34-a6bf-3890b22e4d94 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.760381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992c323-19d6-4d34-a6bf-3890b22e4d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.760830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 289/581] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:08:17 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/038f2ef649844de297237a0c1ec652aa/roles/04a120a50aa648fea07c999a73a1c22c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:08:17.760830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e89fef-57c1-4229-bbb5-854992429642 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:17.778623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6992c323-19d6-4d34-a6bf-3890b22e4d94 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:17.779868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6992c323-19d6-4d34-a6bf-3890b22e4d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:17.872412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 293/582] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:17.890882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e89fef-57c1-4229-bbb5-854992429642 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:17.921303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:17.939332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d947de-1f0c-46b3-a598-71922513658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:17.939957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d947de-1f0c-46b3-a598-71922513658b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:17.943161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d947de-1f0c-46b3-a598-71922513658b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:17.943161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d947de-1f0c-46b3-a598-71922513658b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:17.943161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d947de-1f0c-46b3-a598-71922513658b None admin] RBAC: Adding query filter params (interface=public, service_id=9e53bed5066b4c6ba442a179837b17d0, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:17.943161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d947de-1f0c-46b3-a598-71922513658b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:17.943161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d947de-1f0c-46b3-a598-71922513658b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:17.952493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 290/583] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Oct 24 13:08:17 2022] GET /identity/v3/endpoints?service_id=9e53bed5066b4c6ba442a179837b17d0&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:17.958241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992c323-19d6-4d34-a6bf-3890b22e4d94 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:18.057379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:18.083612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992c323-19d6-4d34-a6bf-3890b22e4d94 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:18.123224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8d61e-2495-42e5-a06a-4c129d60a1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:18.123835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d61e-2495-42e5-a06a-4c129d60a1eb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:18.124113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d61e-2495-42e5-a06a-4c129d60a1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:18.124413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d61e-2495-42e5-a06a-4c129d60a1eb None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:18.125091 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8d61e-2495-42e5-a06a-4c129d60a1eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:18.126504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8d61e-2495-42e5-a06a-4c129d60a1eb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:18.132585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 294/584] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:08:17 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:18.156409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2fc07-4b1e-4795-b6e6-94f5d858cf2e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:18.156699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2fc07-4b1e-4795-b6e6-94f5d858cf2e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:18.156977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2fc07-4b1e-4795-b6e6-94f5d858cf2e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:18.158538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 291/585] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:18.165203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073e5f-e719-4e42-8a13-057200b8c204 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:18.165792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073e5f-e719-4e42-8a13-057200b8c204 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:18.166251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19073e5f-e719-4e42-8a13-057200b8c204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:18.171499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2fc07-4b1e-4795-b6e6-94f5d858cf2e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:18.186148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-19073e5f-e719-4e42-8a13-057200b8c204 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:18.187359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-19073e5f-e719-4e42-8a13-057200b8c204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:18.279854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 295/586] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:18.309309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2fc07-4b1e-4795-b6e6-94f5d858cf2e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:18.350754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:18.351775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:18.352305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:18.352777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] PATH_INFO: `/v3/services/network` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:18.353559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:18.355164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:18.366044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:18.367168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:18.368070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:18.374970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 292/587] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:18 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:18.376962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19073e5f-e719-4e42-8a13-057200b8c204 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:18.457993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:18.484068 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19073e5f-e719-4e42-8a13-057200b8c204 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:18.525153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:18.526139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:18.526739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:18.527228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:18.528141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] RBAC: Adding query filter params (name=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:18.528657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:18.530217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:18.539908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 296/588] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:18.541515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:18.574036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:18.600530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:18.682063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be2918-f331-4622-8687-c2076d1e4f5f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:18.702152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee17a77-dd13-4d75-bcbd-b3eb7a13e736 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:18.718453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:18.719082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:18.719430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:18.719777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:18.720603 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] RBAC: Adding query filter params (type=network) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:18.720868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:18.722377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:18.732251 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 293/589] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:18.756294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:18.756479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5db1e-26a8-4111-b1d8-ad408c95d5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:18.756479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5db1e-26a8-4111-b1d8-ad408c95d5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:18.756983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5db1e-26a8-4111-b1d8-ad408c95d5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:18.759165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5db1e-26a8-4111-b1d8-ad408c95d5b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:18.767861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:18.784510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5db1e-26a8-4111-b1d8-ad408c95d5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-41c5db1e-26a8-4111-b1d8-ad408c95d5b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:18.790170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:18.791103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:18.791945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:18.795659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 297/590] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:18.803635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5db1e-26a8-4111-b1d8-ad408c95d5b0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:18.948322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506d9ca-fb05-47a9-ab84-d312aad7b558 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:18.975033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:18.989310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5db1e-26a8-4111-b1d8-ad408c95d5b0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:19.032950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:19.033551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:19.033828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:19.034096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:19.034748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:19.035048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:19.036432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:19.038292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:19.038885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:19.039186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:19.039449 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:19.040142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] RBAC: Adding query filter params (interface=public, service_id=be0627b32a2445c3b3183dbfb6a56c4b, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:19.040466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:19.041775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:19.050849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 294/591] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Oct 24 13:08:18 2022] GET /identity/v3/endpoints?service_id=be0627b32a2445c3b3183dbfb6a56c4b&interface=public®ion_id=RegionOne => generated 194 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:19.052949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 298/592] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:19.056908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:19.058389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:19.186507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:19.236942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f415f0-44f3-4f76-bc6a-393239fb8192 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:19.267300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324e2b8-53de-4baa-95a1-2f1628dbd79c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:19.304183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:19.305370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:19.305641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:19.305903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:19.327773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d61518-3087-4abe-b49b-fec2af461902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:19.328399 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d61518-3087-4abe-b49b-fec2af461902 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:19.328677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d61518-3087-4abe-b49b-fec2af461902 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:19.328941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d61518-3087-4abe-b49b-fec2af461902 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:19.329586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d61518-3087-4abe-b49b-fec2af461902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:19.331002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d61518-3087-4abe-b49b-fec2af461902 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:19.333312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:19.334747 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:19.340458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 295/593] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:08:19 2022] GET /identity/v3/services => generated 772 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:19.365340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:19.366393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:19.367875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:19.367875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:19.367875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:19.367875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 299/594] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:19.374448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d61518-3087-4abe-b49b-fec2af461902 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:19.454562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:19.543286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d61518-3087-4abe-b49b-fec2af461902 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:19.591065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:19.591670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:19.591947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:19.592507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:19.593140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:19.593439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:19.594863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:19.610071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 296/595] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:19.615534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:19.722750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee59e1-fe41-4cf2-a2dc-85a079c38ded None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:19.733364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:19.764543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:19.765170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:19.765445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:19.765777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] PATH_INFO: `/v3/users/swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:19.776944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:19.778379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:19.794657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:19.795655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:08:19.796684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:19.797729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 300/596] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:19.803846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:19.914729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:19.919156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54874-3b08-4d9c-9c51-0aeebc937e52 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:19.964519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c395e5-a878-42d0-b479-b1e79e857c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:19.965155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c395e5-a878-42d0-b479-b1e79e857c9c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:19.965440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c395e5-a878-42d0-b479-b1e79e857c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:19.965663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c395e5-a878-42d0-b479-b1e79e857c9c None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:19.966404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c395e5-a878-42d0-b479-b1e79e857c9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:19.966707 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c395e5-a878-42d0-b479-b1e79e857c9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:19.968422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c395e5-a878-42d0-b479-b1e79e857c9c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:19.991429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 297/597] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Oct 24 13:08:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:19.997279 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:20.148345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54fa3-e125-4ff6-8879-499074181fda None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:20.190004 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2169535-ca37-4de6-9fb5-e5ffc4a1bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:20.190668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169535-ca37-4de6-9fb5-e5ffc4a1bb75 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:20.190947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169535-ca37-4de6-9fb5-e5ffc4a1bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:20.191214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169535-ca37-4de6-9fb5-e5ffc4a1bb75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:20.247934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:20.284331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2169535-ca37-4de6-9fb5-e5ffc4a1bb75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:20.285788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2169535-ca37-4de6-9fb5-e5ffc4a1bb75 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:20.342565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622763b-9ab3-4860-91d8-bec01d19227e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:20.342948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622763b-9ab3-4860-91d8-bec01d19227e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:20.343351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622763b-9ab3-4860-91d8-bec01d19227e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:20.345017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 298/598] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:20.349720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c1fcf-2cad-4368-bb50-f5214b636640 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:20.350226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c1fcf-2cad-4368-bb50-f5214b636640 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:20.350581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c1fcf-2cad-4368-bb50-f5214b636640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:20.371809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-871c1fcf-2cad-4368-bb50-f5214b636640 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:20.373442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-871c1fcf-2cad-4368-bb50-f5214b636640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:20.450228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 301/599] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:08:20.457005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2169535-ca37-4de6-9fb5-e5ffc4a1bb75 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:20.498095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 299/600] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:20.553258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:20.577296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2169535-ca37-4de6-9fb5-e5ffc4a1bb75 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:20.618206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591ebed-1da1-4f50-834f-8ac3e4c8dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:20.618785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591ebed-1da1-4f50-834f-8ac3e4c8dfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:20.619066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591ebed-1da1-4f50-834f-8ac3e4c8dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:20.619341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591ebed-1da1-4f50-834f-8ac3e4c8dfb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:20.631615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591ebed-1da1-4f50-834f-8ac3e4c8dfb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:20.633165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591ebed-1da1-4f50-834f-8ac3e4c8dfb8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8591ebed-1da1-4f50-834f-8ac3e4c8dfb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:20.649306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:20.650324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:20.651244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:20.651244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:20.651244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:20.651244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 302/601] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:20.657387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c1fcf-2cad-4368-bb50-f5214b636640 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:20.732302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64ca8-a52d-4d36-be60-a897d8d413da None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:20.732936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64ca8-a52d-4d36-be60-a897d8d413da None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:20.732936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb64ca8-a52d-4d36-be60-a897d8d413da None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:20.734162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 303/602] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:20.739173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a78e4-5e2a-47e0-8b8c-58bcfab54b73 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:20.739639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a78e4-5e2a-47e0-8b8c-58bcfab54b73 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:20.739908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a78e4-5e2a-47e0-8b8c-58bcfab54b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:20.766554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-735a78e4-5e2a-47e0-8b8c-58bcfab54b73 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:20.769090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-735a78e4-5e2a-47e0-8b8c-58bcfab54b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:20.796875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:20.820239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c1fcf-2cad-4368-bb50-f5214b636640 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:20.888997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:20.889526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:20.889806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:20.890082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:20.890725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:20.891030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:20.892415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:20.895520 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 304/603] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:20.906507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 300/604] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:20.911721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a78e4-5e2a-47e0-8b8c-58bcfab54b73 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:20.918863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.009084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:21.027264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95baef7-b524-41ed-b97b-18ecaae922dd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.068566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797658a1-baea-41b4-8711-928a8f5d39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.069192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797658a1-baea-41b4-8711-928a8f5d39f3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.069469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797658a1-baea-41b4-8711-928a8f5d39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.069738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797658a1-baea-41b4-8711-928a8f5d39f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.085157 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797658a1-baea-41b4-8711-928a8f5d39f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-797658a1-baea-41b4-8711-928a8f5d39f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:21.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:21.091987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:08:21.093149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:21.094780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a78e4-5e2a-47e0-8b8c-58bcfab54b73 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.097765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 301/605] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:08:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:21.097765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b0a1-6735-49e8-9926-971dfcb542eb None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.098039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b0a1-6735-49e8-9926-971dfcb542eb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.101113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b0a1-6735-49e8-9926-971dfcb542eb None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.101113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 302/606] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:21.102555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84b0a1-6735-49e8-9926-971dfcb542eb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.118472 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:21.136211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404c8af-e314-4f1f-a8b4-c5f9ab8890ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.136987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c8af-e314-4f1f-a8b4-c5f9ab8890ff None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.137383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c8af-e314-4f1f-a8b4-c5f9ab8890ff None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.137751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c8af-e314-4f1f-a8b4-c5f9ab8890ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.144950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404c8af-e314-4f1f-a8b4-c5f9ab8890ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.146521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404c8af-e314-4f1f-a8b4-c5f9ab8890ff None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c404c8af-e314-4f1f-a8b4-c5f9ab8890ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:21.158341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:21.159436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:21.160611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:21.160611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:21.160611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:21.160611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:21.160611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:21.160611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:21.160611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:21.160611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:21.162266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 305/607] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:21.163575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffd44-6154-495f-b3b9-f8909c9d896b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.164602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffd44-6154-495f-b3b9-f8909c9d896b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.164944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ffd44-6154-495f-b3b9-f8909c9d896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.182463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5b2ffd44-6154-495f-b3b9-f8909c9d896b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:21.183697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5b2ffd44-6154-495f-b3b9-f8909c9d896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:21.227458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84b0a1-6735-49e8-9926-971dfcb542eb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.256194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 306/608] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:21.257306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ffd44-6154-495f-b3b9-f8909c9d896b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.268614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.269736 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.270085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.270418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.271140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:21.271510 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.274205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.284728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 303/609] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:21.290249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.315243 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:21.374198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ffd44-6154-495f-b3b9-f8909c9d896b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.387932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:21.411055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.411790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.412173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.412742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.413571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:21.414011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.416708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.429944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3fcd1-c0aa-432b-9056-93b2c718546b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.434615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 307/610] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:21.435641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.471050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.471941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.472227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.472531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.483507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.485109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:21.500855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:21.501842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:21.502751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:21.502751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:21.502751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:21.502751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 304/611] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:21.502751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.555800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd960f-b284-4569-801a-640262f08bc0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.557381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:21.590646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.591769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.592671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.592946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.593525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.594898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:21.603704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:21.604721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:21.605647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:21.609686 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 308/612] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:21.610515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.624072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b93e9-5d52-468c-b24b-686ec3626e15 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.647808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:21.660058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.661636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.661636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.661636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.662048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=035a080de9d54a56af29cc56b33a1ddc) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:21.662364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.663785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.669848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 305/613] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/role_assignments?user.id=035a080de9d54a56af29cc56b33a1ddc&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:21.670642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.755027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5b8d5-ef5e-4211-aa42-ecb9d021ccd7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.791872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.792571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.792895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.793170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.794116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:21.794116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.795540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.808023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43be7c-5234-4e0b-aeb3-cbff5454ef7d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.808918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 309/614] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:21.809889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.852896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:21.854079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.854655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.857218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.857218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.857218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.857218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:21.869598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:21.870862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:08:21.871924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:21.876418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 306/615] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:21.876418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:21.935568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21260-22f4-45e3-b500-127f4502b476 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:21.937550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:21.977233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:21.978105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:21.978380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:21.978645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:21.979416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:21.979717 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:21.981402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:21.987521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 310/616] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:21.989082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:22.004088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342ccc-6023-41e1-9f87-12b3309f9256 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:22.043629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:22.044254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.044545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.044836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.053466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:22.055471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:22.067070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:22.068052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 307/617] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:22.069376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:22.106226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:22.117794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61beb-28c9-48e7-8438-4afd54df8081 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:22.156290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea98449-be8e-40db-ab7d-fa6b82301cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:22.157269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea98449-be8e-40db-ab7d-fa6b82301cdd None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.157269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea98449-be8e-40db-ab7d-fa6b82301cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.157557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea98449-be8e-40db-ab7d-fa6b82301cdd None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.158087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea98449-be8e-40db-ab7d-fa6b82301cdd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:22.158386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea98449-be8e-40db-ab7d-fa6b82301cdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:22.159730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea98449-be8e-40db-ab7d-fa6b82301cdd None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:22.169290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 311/618] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Oct 24 13:08:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:22.169290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f974-93cb-4a71-9b2d-ae7fd996d644 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.169290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f974-93cb-4a71-9b2d-ae7fd996d644 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.169290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f974-93cb-4a71-9b2d-ae7fd996d644 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.169290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 312/619] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:22.169290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44f974-93cb-4a71-9b2d-ae7fd996d644 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:22.192299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7febd-1809-4bd9-97f7-409219c3d35b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:22.208079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:22.232968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:22.233798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.234232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.234507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.235129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:22.235432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:22.236823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:22.248931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 308/620] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:22.248931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:22.294351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44f974-93cb-4a71-9b2d-ae7fd996d644 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:22.337959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae28f0-455e-42c4-b58f-77c51a6e5982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:22.338619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae28f0-455e-42c4-b58f-77c51a6e5982 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.338899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae28f0-455e-42c4-b58f-77c51a6e5982 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.339174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae28f0-455e-42c4-b58f-77c51a6e5982 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.339931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae28f0-455e-42c4-b58f-77c51a6e5982 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:22.340247 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae28f0-455e-42c4-b58f-77c51a6e5982 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:22.341856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae28f0-455e-42c4-b58f-77c51a6e5982 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:22.363149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 313/621] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Oct 24 13:08:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:22.364663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5289-0229-4c0b-8158-5ccff26e360f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.365139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5289-0229-4c0b-8158-5ccff26e360f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.365636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb5289-0229-4c0b-8158-5ccff26e360f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.369865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c49a41-1e69-40d7-9b15-33c5f59bbec2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:22.386734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-60eb5289-0229-4c0b-8158-5ccff26e360f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:22.388020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-60eb5289-0229-4c0b-8158-5ccff26e360f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:22.397507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:22.412837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:22.413459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.413734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.414046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.416046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:22.416390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:22.417866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:22.425161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 309/622] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Oct 24 13:08:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:22.426044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:22.478586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:22.479431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 314/623] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:22.480246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5289-0229-4c0b-8158-5ccff26e360f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:22.584024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10e24f-f00e-4337-b852-391c599dede8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:22.655976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f2900-7cd2-4880-9f33-005c0912f20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:22.657056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f2900-7cd2-4880-9f33-005c0912f20c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.657590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f2900-7cd2-4880-9f33-005c0912f20c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.658054 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f2900-7cd2-4880-9f33-005c0912f20c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.678650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb5289-0229-4c0b-8158-5ccff26e360f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:22.741741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dcaa4-52d3-45e8-b2f3-f10e92864f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:22.742716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dcaa4-52d3-45e8-b2f3-f10e92864f77 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:22.743963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dcaa4-52d3-45e8-b2f3-f10e92864f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:22.744457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dcaa4-52d3-45e8-b2f3-f10e92864f77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:22.745313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dcaa4-52d3-45e8-b2f3-f10e92864f77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:22.747578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dcaa4-52d3-45e8-b2f3-f10e92864f77 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:22.778715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f2900-7cd2-4880-9f33-005c0912f20c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:22.780256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f2900-7cd2-4880-9f33-005c0912f20c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:22.821519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:22.856317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:22.872611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 315/624] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:22.874356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dcaa4-52d3-45e8-b2f3-f10e92864f77 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:22.903538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 310/625] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:08:22.911698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f2900-7cd2-4880-9f33-005c0912f20c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:23.122455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f2900-7cd2-4880-9f33-005c0912f20c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:23.128472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dcaa4-52d3-45e8-b2f3-f10e92864f77 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:23.172496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:23.173426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:23.173711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:23.173986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:23.178153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:23.179037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:23.180065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:23.180570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] PATH_INFO: `/v3/services/network` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:23.181346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:23.182904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:23.183626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:23.188421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:23.189873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:23.194765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:23.201318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:23.202193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 316/626] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:22 2022] GET /identity/v3/services/network => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:23.207993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:23.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:23.212012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:23.213341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:23.213341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:23.213341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:23.213341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 311/627] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:23.218616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:23.279008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:23.325403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e6a6a-5e82-4cb3-adb6-c618e4b6696a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:23.336960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964cde-64ea-42db-bd89-73efe8e2d910 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:23.363197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:23.364116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:23.364699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:23.365310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:23.366227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:23.366735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:23.369066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:23.374180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 317/628] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:23.380388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:23.381807 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:23.382423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:23.383266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:23.383534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:23.384180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:23.384526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:23.385969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:23.398956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 312/629] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:23.404593 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:23.493775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:23.563502 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447319c9-22fd-4635-ad2e-eff7631e28e5 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:23.565867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59587959-9378-4165-91a9-7e87b9c7a95c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:23.594665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:23.603113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:23.603777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:23.604049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:23.609461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:23.611265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:23.612744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:23.625184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:23.626165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:23.627439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 313/630] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:23.633359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:23.637561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:23.638175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:23.638455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:23.638727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:23.639356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] RBAC: Adding query filter params (type=network) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:23.639658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:23.641042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:23.656263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 318/631] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:23.661182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:23.740191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01977b12-b0c4-4f12-b1bc-15ad15f73eb8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:23.761699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:23.782031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:23.782634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:23.783809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:23.784091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:23.786701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:23.786701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:23.786701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:23.809379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 314/632] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:23.815228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:23.868628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8ddb3-f9eb-4721-81fa-4aecddb53abf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:23.941163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1d12d-1e4d-4664-83ec-cdc1d155fdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:23.942084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1d12d-1e4d-4664-83ec-cdc1d155fdb2 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:23.942366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1d12d-1e4d-4664-83ec-cdc1d155fdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:23.942637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1d12d-1e4d-4664-83ec-cdc1d155fdb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:23.943305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1d12d-1e4d-4664-83ec-cdc1d155fdb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:23.944724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1d12d-1e4d-4664-83ec-cdc1d155fdb2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:23.984014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79997fba-ff59-43fc-90ba-73ad9c784930 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:24.018573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf7da5-a857-4adb-a99e-6309d86abd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:24.019181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf7da5-a857-4adb-a99e-6309d86abd46 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:24.019462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf7da5-a857-4adb-a99e-6309d86abd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:24.019740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf7da5-a857-4adb-a99e-6309d86abd46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:24.020487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf7da5-a857-4adb-a99e-6309d86abd46 None admin] RBAC: Adding query filter params (role.id=04a120a50aa648fea07c999a73a1c22c, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=038f2ef649844de297237a0c1ec652aa) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:24.020795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf7da5-a857-4adb-a99e-6309d86abd46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:24.022220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf7da5-a857-4adb-a99e-6309d86abd46 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:24.029368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 315/633] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:23 2022] GET /identity/v3/role_assignments?user.id=038f2ef649844de297237a0c1ec652aa&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=04a120a50aa648fea07c999a73a1c22c => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:24.038245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cd1e8-26ac-4a93-96a5-bd07c54c4c82 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:24.039525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cd1e8-26ac-4a93-96a5-bd07c54c4c82 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:24.039831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 319/634] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:23 2022] POST /identity/v3/endpoints => generated 337 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:24.040375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cd1e8-26ac-4a93-96a5-bd07c54c4c82 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:24.042614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 316/635] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:24.049478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41564b02-2d43-4f2a-ab7c-3bee91645532 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:24.050094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41564b02-2d43-4f2a-ab7c-3bee91645532 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:24.054191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41564b02-2d43-4f2a-ab7c-3bee91645532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:24.074507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-41564b02-2d43-4f2a-ab7c-3bee91645532 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:24.076439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-41564b02-2d43-4f2a-ab7c-3bee91645532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:24.153790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:24.214419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 320/636] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:24.507299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cd1e8-26ac-4a93-96a5-bd07c54c4c82 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:24.544772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:24.611187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cd1e8-26ac-4a93-96a5-bd07c54c4c82 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:24.648531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddecabf-02d0-4078-9b23-e466ec29c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:24.649237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddecabf-02d0-4078-9b23-e466ec29c4ce None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:24.649511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddecabf-02d0-4078-9b23-e466ec29c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:24.649780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddecabf-02d0-4078-9b23-e466ec29c4ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:24.650414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddecabf-02d0-4078-9b23-e466ec29c4ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:24.651789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddecabf-02d0-4078-9b23-e466ec29c4ce None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-cddecabf-02d0-4078-9b23-e466ec29c4ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:24.661435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:24.662590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:24.663588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 317/637] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:24.668589 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41564b02-2d43-4f2a-ab7c-3bee91645532 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:24.770943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce299a-5963-4757-b0c4-4e2ade7a4c9f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:24.771118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce299a-5963-4757-b0c4-4e2ade7a4c9f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:24.771400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce299a-5963-4757-b0c4-4e2ade7a4c9f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:24.773774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 318/638] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:24.779065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389730c-1b89-4e95-9f0a-d67eb418391a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:24.779463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389730c-1b89-4e95-9f0a-d67eb418391a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:24.779761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389730c-1b89-4e95-9f0a-d67eb418391a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:24.781987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41564b02-2d43-4f2a-ab7c-3bee91645532 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:24.798320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c389730c-1b89-4e95-9f0a-d67eb418391a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:24.799665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c389730c-1b89-4e95-9f0a-d67eb418391a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:24.826115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:24.827282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:24.828186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:24.828855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:24.829738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:24.830424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:24.832593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:24.839007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 321/639] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:24.848385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:24.873574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 319/640] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:24.888658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:24.957801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc005a-af44-481c-8cc0-9a29f50875ad None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:24.994048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762da67-cf64-4d26-8d2f-1f9cbed55147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:24.994659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762da67-cf64-4d26-8d2f-1f9cbed55147 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:24.994942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762da67-cf64-4d26-8d2f-1f9cbed55147 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:24.995215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762da67-cf64-4d26-8d2f-1f9cbed55147 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:24.995844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762da67-cf64-4d26-8d2f-1f9cbed55147 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:24.996150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762da67-cf64-4d26-8d2f-1f9cbed55147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:24.997623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762da67-cf64-4d26-8d2f-1f9cbed55147 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:25.002559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 322/641] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:25.174771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389730c-1b89-4e95-9f0a-d67eb418391a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:25.205095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:25.282889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389730c-1b89-4e95-9f0a-d67eb418391a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:25.320803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26876-f39c-4b08-a1e7-eb4feb71803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:25.321841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26876-f39c-4b08-a1e7-eb4feb71803f None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:25.322324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26876-f39c-4b08-a1e7-eb4feb71803f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:25.322806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26876-f39c-4b08-a1e7-eb4feb71803f None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:25.323661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd26876-f39c-4b08-a1e7-eb4feb71803f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:25.325270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd26876-f39c-4b08-a1e7-eb4feb71803f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:25.389058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 320/642] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:08:25 2022] POST /identity/v3/services => generated 269 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:25.604456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711433-82b4-418b-8403-4ea7d2131b9c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:25.604944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711433-82b4-418b-8403-4ea7d2131b9c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:25.605323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711433-82b4-418b-8403-4ea7d2131b9c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:25.606932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 323/643] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:25.612870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f87b0f-fbb5-42a9-9e85-8d8eb897a916 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:25.612870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f87b0f-fbb5-42a9-9e85-8d8eb897a916 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:25.613412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f87b0f-fbb5-42a9-9e85-8d8eb897a916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:25.631620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-64f87b0f-fbb5-42a9-9e85-8d8eb897a916 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:25.633016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-64f87b0f-fbb5-42a9-9e85-8d8eb897a916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:25.716383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 321/644] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:25 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:25.985124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a711433-82b4-418b-8403-4ea7d2131b9c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:26.096627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a711433-82b4-418b-8403-4ea7d2131b9c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:26.098350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:26.133649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:26.134364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:26.134651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:26.134917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:26.135496 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:26.136891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:26.146035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:26.147063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:26.148995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 324/645] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:26.158134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f87b0f-fbb5-42a9-9e85-8d8eb897a916 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:26.172746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:26.261516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f87b0f-fbb5-42a9-9e85-8d8eb897a916 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:26.297962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5607284-d275-485b-8bfe-e3daf497e643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:26.298570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5607284-d275-485b-8bfe-e3daf497e643 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:26.298846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5607284-d275-485b-8bfe-e3daf497e643 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:26.299116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5607284-d275-485b-8bfe-e3daf497e643 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:26.299746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5607284-d275-485b-8bfe-e3daf497e643 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:26.300045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5607284-d275-485b-8bfe-e3daf497e643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:26.301440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5607284-d275-485b-8bfe-e3daf497e643 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:26.306549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 322/646] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:26.312166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:26.426904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c70d9-e695-4c52-84d4-7e5b9ab24416 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:26.463330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4b6d0-c79f-4aa2-8350-5f7b6b7f745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:26.464487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b6d0-c79f-4aa2-8350-5f7b6b7f745f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:26.464998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b6d0-c79f-4aa2-8350-5f7b6b7f745f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:26.465540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b6d0-c79f-4aa2-8350-5f7b6b7f745f None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:26.466381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4b6d0-c79f-4aa2-8350-5f7b6b7f745f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:26.467125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4b6d0-c79f-4aa2-8350-5f7b6b7f745f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:26.468899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4b6d0-c79f-4aa2-8350-5f7b6b7f745f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:26.478044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 325/647] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:26.501286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ac19d-d418-4af3-b246-778ff9f833f5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:26.501615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ac19d-d418-4af3-b246-778ff9f833f5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:26.501837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ac19d-d418-4af3-b246-778ff9f833f5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:26.503535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 323/648] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:26.509663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d63c-83b0-4d4f-8bae-7240be238b3f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:26.510196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d63c-83b0-4d4f-8bae-7240be238b3f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:26.510622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70d63c-83b0-4d4f-8bae-7240be238b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:26.530668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9b70d63c-83b0-4d4f-8bae-7240be238b3f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:26.532131 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9b70d63c-83b0-4d4f-8bae-7240be238b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:26.553006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:26.611080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 326/649] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:26 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:26.879369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ac19d-d418-4af3-b246-778ff9f833f5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:26.965379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:26.988733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ac19d-d418-4af3-b246-778ff9f833f5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:27.024423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6baf2e-81a5-403a-bbcc-cd658f280d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:27.025383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6baf2e-81a5-403a-bbcc-cd658f280d74 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.025852 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6baf2e-81a5-403a-bbcc-cd658f280d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.026315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6baf2e-81a5-403a-bbcc-cd658f280d74 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.036679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6baf2e-81a5-403a-bbcc-cd658f280d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9a6baf2e-81a5-403a-bbcc-cd658f280d74 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:27.042265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:27.043271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:08:27.044143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:27.046376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 324/650] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Oct 24 13:08:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:27.053641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70d63c-83b0-4d4f-8bae-7240be238b3f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:27.161858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70d63c-83b0-4d4f-8bae-7240be238b3f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:27.200413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd93b18-242e-41ad-b508-45e8f85fa286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:27.201012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd93b18-242e-41ad-b508-45e8f85fa286 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.201292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd93b18-242e-41ad-b508-45e8f85fa286 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.201566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd93b18-242e-41ad-b508-45e8f85fa286 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.202224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd93b18-242e-41ad-b508-45e8f85fa286 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:27.202523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd93b18-242e-41ad-b508-45e8f85fa286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:27.203873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd93b18-242e-41ad-b508-45e8f85fa286 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:27.213262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 327/651] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Oct 24 13:08:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:27.218817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6baf2e-81a5-403a-bbcc-cd658f280d74 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:27.277404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfe240-6ff1-4d0c-8087-cc24e1065508 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.277685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfe240-6ff1-4d0c-8087-cc24e1065508 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.277960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfe240-6ff1-4d0c-8087-cc24e1065508 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.279405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 328/652] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:27.286416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e74d3-e49c-45e6-aeb4-9d53367f59bf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.287258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e74d3-e49c-45e6-aeb4-9d53367f59bf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.287754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e74d3-e49c-45e6-aeb4-9d53367f59bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.311115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0e6e74d3-e49c-45e6-aeb4-9d53367f59bf None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:27.313403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0e6e74d3-e49c-45e6-aeb4-9d53367f59bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:27.324604 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6baf2e-81a5-403a-bbcc-cd658f280d74 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:27.332632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:27.362456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:27.363207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.363555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.364631 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.374636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:27.378705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:27.380163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:27.402921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:27.403971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:27.404957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:27.404957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:27.404957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:27.404957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 325/653] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:27.404957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:27.405790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 329/654] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:27 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:27.514100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2e4ea-55cf-42ed-ac91-800155e73da5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:27.549471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:27.550072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.550351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.550624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.551259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:27.551562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:27.552947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:27.565663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 326/655] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:27.571852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e74d3-e49c-45e6-aeb4-9d53367f59bf None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:27.681198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e74d3-e49c-45e6-aeb4-9d53367f59bf None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:27.705633 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:27.718073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:27.718919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.719354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.719776 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.727992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:27.729706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:27.742116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:27.743086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:08:27.744077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:27.746674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 330/656] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Oct 24 13:08:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:27.753966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:27.764388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:27.765355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:27.825397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2081e9-5fbe-42ef-96af-0551dd8aa1b4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:27.861919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f386e1-0684-42f5-b320-ef479c292efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:27.862520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f386e1-0684-42f5-b320-ef479c292efc None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.862785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f386e1-0684-42f5-b320-ef479c292efc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.863054 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f386e1-0684-42f5-b320-ef479c292efc None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.863689 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f386e1-0684-42f5-b320-ef479c292efc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:27.864104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f248-ed59-4f80-98bf-8152cf8e5df4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:27.865071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f386e1-0684-42f5-b320-ef479c292efc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:27.897375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68776b-f66f-4ffd-b607-fb6b01673814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:27.898473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68776b-f66f-4ffd-b607-fb6b01673814 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.898745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68776b-f66f-4ffd-b607-fb6b01673814 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.899014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68776b-f66f-4ffd-b607-fb6b01673814 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.899756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68776b-f66f-4ffd-b607-fb6b01673814 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:27.900060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68776b-f66f-4ffd-b607-fb6b01673814 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:27.901508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68776b-f66f-4ffd-b607-fb6b01673814 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:27.918787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 331/657] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Oct 24 13:08:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:27.920046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498c302-fc7a-42f3-a051-fe5088b67216 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.920350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498c302-fc7a-42f3-a051-fe5088b67216 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.920670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498c302-fc7a-42f3-a051-fe5088b67216 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.921993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 332/658] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:27.924145 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498c302-fc7a-42f3-a051-fe5088b67216 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:27.964670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 327/659] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:08:27 2022] POST /identity/v3/services => generated 241 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:27.966042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8e48-035e-4dd5-8cdb-62798f324939 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:27.966331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8e48-035e-4dd5-8cdb-62798f324939 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:27.966657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8e48-035e-4dd5-8cdb-62798f324939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:27.987667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-364d8e48-035e-4dd5-8cdb-62798f324939 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:27.989120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-364d8e48-035e-4dd5-8cdb-62798f324939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:28.004653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:28.057564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498c302-fc7a-42f3-a051-fe5088b67216 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:28.071341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 328/660] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:27 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:28.099893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:28.100574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.100854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.101460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.113411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:28.114957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:28.136765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:28.137830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:28.138736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:28.138736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:28.138736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:28.141061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 333/661] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:28.149765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d8e48-035e-4dd5-8cdb-62798f324939 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:28.197755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:28.263737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d8e48-035e-4dd5-8cdb-62798f324939 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:28.300153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:28.300788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.301109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.301386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.302023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:28.302324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:28.303738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:28.316462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 329/662] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:28.322485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:28.384213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:28.428922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd36ec-c0c0-41ea-837f-ec8e121be505 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:28.442784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:28.470250 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:28.471221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.471554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.471865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.481060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:28.483160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:28.490387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:28.495764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ea4b0-a9fd-43ce-9f8a-b70d5e8001a3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:28.503389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:28.504597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:28.505513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:28.505513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:28.505513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:28.505513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:28.505513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:28.505513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:28.505513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:28.505513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:28.508871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 334/663] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:28.515092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:28.538514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:28.539266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.539549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.539818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.540432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:28.542119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:28.553061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:28.554038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:28.555496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:28.559869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 330/664] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Oct 24 13:08:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:28.560927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:28.622781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0e823-14b9-4976-b626-d7a29c60fe2a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:28.670468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:28.671645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.672067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.672497 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.673635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:28.673950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:28.675796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:28.690032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee440f-e8c5-4826-868f-2d8020f0a4c0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:28.693402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 335/665] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:28.701202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:28.734720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc5bd3-a8d3-4901-af12-8d15c4a612ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:28.735497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5bd3-a8d3-4901-af12-8d15c4a612ac None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.736045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5bd3-a8d3-4901-af12-8d15c4a612ac None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.736456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc5bd3-a8d3-4901-af12-8d15c4a612ac None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.737525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc5bd3-a8d3-4901-af12-8d15c4a612ac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:28.737938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc5bd3-a8d3-4901-af12-8d15c4a612ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:28.739639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc5bd3-a8d3-4901-af12-8d15c4a612ac None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:28.740511 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:28.746592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 331/666] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:28.753740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376c2b-185a-4591-8e01-8a31c9cf4f7e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.753740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376c2b-185a-4591-8e01-8a31c9cf4f7e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.753740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe376c2b-185a-4591-8e01-8a31c9cf4f7e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.753740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 332/667] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:28.753740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe376c2b-185a-4591-8e01-8a31c9cf4f7e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:28.784734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:28.815894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7baeeb3-0d55-4841-b78b-ef8bdf048f18 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:28.852397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8104b-e52d-4f91-b15f-74f4ef50beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:28.853162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8104b-e52d-4f91-b15f-74f4ef50beb1 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.853439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8104b-e52d-4f91-b15f-74f4ef50beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.853711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8104b-e52d-4f91-b15f-74f4ef50beb1 None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/038f2ef649844de297237a0c1ec652aa/roles/8d9d7fa07c94446e82a45b539babe09d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.856069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8104b-e52d-4f91-b15f-74f4ef50beb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=038f2ef649844de297237a0c1ec652aa, role_id=8d9d7fa07c94446e82a45b539babe09d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:28.857538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8104b-e52d-4f91-b15f-74f4ef50beb1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:28.859336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe376c2b-185a-4591-8e01-8a31c9cf4f7e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:28.893424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fae81-8fbf-4ec9-b76c-79bcd21dc0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:28.893957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fae81-8fbf-4ec9-b76c-79bcd21dc0ac None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.894232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fae81-8fbf-4ec9-b76c-79bcd21dc0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.894516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fae81-8fbf-4ec9-b76c-79bcd21dc0ac None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.895137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fae81-8fbf-4ec9-b76c-79bcd21dc0ac None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:28.895437 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fae81-8fbf-4ec9-b76c-79bcd21dc0ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:28.896825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fae81-8fbf-4ec9-b76c-79bcd21dc0ac None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:28.902107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 333/668] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:28.903217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aab13-c1e8-46f1-8667-bca39c93a679 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:28.903500 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aab13-c1e8-46f1-8667-bca39c93a679 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:28.903771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9aab13-c1e8-46f1-8667-bca39c93a679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:28.921985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8b9aab13-c1e8-46f1-8667-bca39c93a679 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:28.923162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8b9aab13-c1e8-46f1-8667-bca39c93a679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:28.932593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 336/669] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:08:28 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/038f2ef649844de297237a0c1ec652aa/roles/8d9d7fa07c94446e82a45b539babe09d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:08:28.932593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8104b-e52d-4f91-b15f-74f4ef50beb1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:29.012147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 334/670] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:28 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:29.049248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:29.060298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8104b-e52d-4f91-b15f-74f4ef50beb1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:29.106378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:29.107351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:29.107833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:29.108479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:29.109441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] RBAC: Adding query filter params (interface=public, service_id=df331fc0720a419399b3f47b09e1f5c0, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:29.109938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:29.111611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:29.120863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 337/671] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Oct 24 13:08:28 2022] GET /identity/v3/endpoints?service_id=df331fc0720a419399b3f47b09e1f5c0&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:29.129083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9aab13-c1e8-46f1-8667-bca39c93a679 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:29.158771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:29.237752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9aab13-c1e8-46f1-8667-bca39c93a679 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:29.278056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45e1ac-a028-4dd0-be94-e38e51ccc05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:29.279009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e1ac-a028-4dd0-be94-e38e51ccc05e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:29.279537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e1ac-a028-4dd0-be94-e38e51ccc05e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:29.280008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e1ac-a028-4dd0-be94-e38e51ccc05e None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:29.280841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45e1ac-a028-4dd0-be94-e38e51ccc05e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:29.282415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45e1ac-a028-4dd0-be94-e38e51ccc05e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:29.289175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 335/672] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:08:29 2022] GET /identity/v3/services => generated 1258 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:08:29.297598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:29.411211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356d7a4-cf44-40f1-8ca5-4e1e07bfcdd0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:29.449909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0b8eb-62f6-4555-95fa-33632d073e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:29.451036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0b8eb-62f6-4555-95fa-33632d073e57 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:29.451577 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0b8eb-62f6-4555-95fa-33632d073e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:29.452038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0b8eb-62f6-4555-95fa-33632d073e57 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:29.452920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0b8eb-62f6-4555-95fa-33632d073e57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:29.454533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0b8eb-62f6-4555-95fa-33632d073e57 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:29.559144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 338/673] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:08:29 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:30.239708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36cd8-3014-436c-847f-e4989eacec94 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:30.240215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36cd8-3014-436c-847f-e4989eacec94 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:30.240326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db36cd8-3014-436c-847f-e4989eacec94 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:30.241906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 336/674] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:30.247504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92353-027c-4787-b427-1cd08d54151d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:30.247934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92353-027c-4787-b427-1cd08d54151d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:30.248134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92353-027c-4787-b427-1cd08d54151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:30.266918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-60a92353-027c-4787-b427-1cd08d54151d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:30.268130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-60a92353-027c-4787-b427-1cd08d54151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:30.280459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:30.348491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 339/675] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:30.621595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db36cd8-3014-436c-847f-e4989eacec94 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:30.709096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:30.726817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db36cd8-3014-436c-847f-e4989eacec94 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:30.763002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:30.763948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:30.764486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:30.764975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:30.765752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:30.767310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:30.776979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:30.777985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:30.778869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:30.781611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 337/676] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:30.790241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a92353-027c-4787-b427-1cd08d54151d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:30.891243 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a92353-027c-4787-b427-1cd08d54151d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:30.927859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:30.928504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:30.928782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:30.929058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:30.929719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:30.930029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:30.931389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:30.936654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 340/677] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:30.942193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:31.050771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c0d4-f515-4510-994f-f2ecd5aba473 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:31.086056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:31.086989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.087487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.087959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.089234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:31.089745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:31.091311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:31.097432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 338/678] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:31.103234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:31.115025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:31.213045 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacf21e-e27a-4200-8be5-5d92be50c883 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:31.250135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75e8b-7543-4a24-8ec5-924a773dc4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:31.250768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75e8b-7543-4a24-8ec5-924a773dc4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.251021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75e8b-7543-4a24-8ec5-924a773dc4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.251271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75e8b-7543-4a24-8ec5-924a773dc4e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.251967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75e8b-7543-4a24-8ec5-924a773dc4e3 None admin] RBAC: Adding query filter params (interface=public, service_id=60e16e9ffb804708ba7e4697d1a38a25, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:31.252295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75e8b-7543-4a24-8ec5-924a773dc4e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:31.253713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75e8b-7543-4a24-8ec5-924a773dc4e3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:31.264281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 341/679] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Oct 24 13:08:31 2022] GET /identity/v3/endpoints?service_id=60e16e9ffb804708ba7e4697d1a38a25&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:31.265629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:31.285179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:31.327701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6617b-47e8-4dec-88f8-42f9aad030d4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.328563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6617b-47e8-4dec-88f8-42f9aad030d4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.328864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6617b-47e8-4dec-88f8-42f9aad030d4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.330219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 342/680] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:31.335701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deb06e-e5eb-4702-84fc-06d1478a4004 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.336159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deb06e-e5eb-4702-84fc-06d1478a4004 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.336538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deb06e-e5eb-4702-84fc-06d1478a4004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.356075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-33deb06e-e5eb-4702-84fc-06d1478a4004 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:31.357564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-33deb06e-e5eb-4702-84fc-06d1478a4004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:31.381824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cac13d-458e-4423-9d8d-55134d1a14c7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:31.423560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5700ad6-490e-49d4-9a9e-de2139f4674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:31.424161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700ad6-490e-49d4-9a9e-de2139f4674f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.424469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700ad6-490e-49d4-9a9e-de2139f4674f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.424744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700ad6-490e-49d4-9a9e-de2139f4674f None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.425347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5700ad6-490e-49d4-9a9e-de2139f4674f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:31.426695 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5700ad6-490e-49d4-9a9e-de2139f4674f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:31.434275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:31.434423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 339/681] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:08:31 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:08:31.444176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 343/682] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:31.487384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec8660-79e5-473c-b6b4-1eed50584ffc None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.487560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec8660-79e5-473c-b6b4-1eed50584ffc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.487840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec8660-79e5-473c-b6b4-1eed50584ffc None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.489541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 340/683] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:31.495893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817400b-7222-450f-a22f-13a0c38f513f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.496214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817400b-7222-450f-a22f-13a0c38f513f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.496483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817400b-7222-450f-a22f-13a0c38f513f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.516415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8817400b-7222-450f-a22f-13a0c38f513f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:31.517618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8817400b-7222-450f-a22f-13a0c38f513f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:31.597219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 344/684] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:31.726878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec8660-79e5-473c-b6b4-1eed50584ffc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:31.815951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:31.838435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec8660-79e5-473c-b6b4-1eed50584ffc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:31.860022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a5ab-13ee-4cf3-9a1e-e3c7c498de3b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.860356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a5ab-13ee-4cf3-9a1e-e3c7c498de3b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.860643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a5ab-13ee-4cf3-9a1e-e3c7c498de3b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.862112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 345/685] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:31.868649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eae71-c600-4522-98d6-1c792cba97d8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.868992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eae71-c600-4522-98d6-1c792cba97d8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.869276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075eae71-c600-4522-98d6-1c792cba97d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.878884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898bec7-cf6b-45a1-8927-cfcf895448f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:31.879815 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898bec7-cf6b-45a1-8927-cfcf895448f7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:31.880325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898bec7-cf6b-45a1-8927-cfcf895448f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:31.880799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898bec7-cf6b-45a1-8927-cfcf895448f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:31.887340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-075eae71-c600-4522-98d6-1c792cba97d8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:31.888715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-075eae71-c600-4522-98d6-1c792cba97d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:31.892456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898bec7-cf6b-45a1-8927-cfcf895448f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d898bec7-cf6b-45a1-8927-cfcf895448f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:31.897522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:31.898740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:08:31.899813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:31.901828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 341/686] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Oct 24 13:08:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:31.902762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898bec7-cf6b-45a1-8927-cfcf895448f7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:31.969492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 346/687] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:31.970556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075eae71-c600-4522-98d6-1c792cba97d8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.009626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898bec7-cf6b-45a1-8927-cfcf895448f7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.050416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.051408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.051763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.052107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.053197 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.054905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.056561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:32.067044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:32.068044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:32.069072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 342/688] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Oct 24 13:08:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:32.073942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.080691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:32.120749 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075eae71-c600-4522-98d6-1c792cba97d8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.158113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.158852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.159229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.159600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.160390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:32.160806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.162285 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.171577 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 347/689] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Oct 24 13:08:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:32.177118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.213293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41b5b9-1c2c-40e3-a735-c295959f5094 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.254629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.255239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.255523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.255797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.256472 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:32.256779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.258171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.264388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 343/690] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:32.264388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.286347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90882e-c65f-4a12-b9a5-c18e1b779b44 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.311571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:32.323888 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.324484 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.324765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.325038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.336318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.337758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.338618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:32.353434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:32.356388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:32.357370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:32.357370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:32.357370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:32.357370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 348/691] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:32.357370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.386978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5e25e-3ac3-44c0-83f8-31e431415296 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.424442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.425061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.425341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.425613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.426187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.427778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:32.436800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:32.437873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:32.438735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 344/692] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:32.439559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.463657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dba72f-6fbc-4ae7-88d5-ec51f2df45d2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.497522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.498121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.498401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.498673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.499296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:32.499594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.500977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.505619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 349/693] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:32.506392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.550653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:32.557506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e7e84-0773-443a-a157-abb65a24431a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.589359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:32.595762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.596384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.596669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.596942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.597567 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:32.597875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.599216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.615091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 345/694] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:32.615091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.615349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341194b4-98b6-4c99-88ba-cc650b3221d4 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.649096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.649699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.649980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.650256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.650873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:32.651179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.652571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.657390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 350/695] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:32.657986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.722140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29257a4-8c4d-4881-9380-5be30a866daf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.756396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e14db3-1c58-44b0-9df4-eb1e89a7f03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.756964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14ac6-16aa-49c2-b19e-5617585d70b3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.757087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14db3-1c58-44b0-9df4-eb1e89a7f03d None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.757798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14db3-1c58-44b0-9df4-eb1e89a7f03d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.758071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e14db3-1c58-44b0-9df4-eb1e89a7f03d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.758734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e14db3-1c58-44b0-9df4-eb1e89a7f03d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.759190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:32.760864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e14db3-1c58-44b0-9df4-eb1e89a7f03d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.793188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.793814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.794091 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:32.794358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:32.801155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:32.802597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:32.815106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:32.816160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 351/696] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:32.817258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.845236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 346/697] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:32.845986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e14db3-1c58-44b0-9df4-eb1e89a7f03d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:32.908091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:32.956752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95995652-512e-4b39-9991-bf2891a14d22 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.987150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e14db3-1c58-44b0-9df4-eb1e89a7f03d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:32.998300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:32.999284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:32.999800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.000357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.001211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:33.001821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:33.003449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:33.010444 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 352/698] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:33.017361 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:33.035667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:33.036614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.037151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.037618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.038622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:33.039164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:33.040826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:33.053031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:33.063497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 347/699] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Oct 24 13:08:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:33.069199 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:33.144171 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80834a3-89a1-4c65-af8e-244f2716c4bf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:33.178914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:33.180428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.180810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.181175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.181964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] RBAC: Adding query filter params (interface=public, service_id=a45b06a84ee14139aeb97c5a0c3a47af, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:33.182367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:33.184042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:33.189720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 353/700] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Oct 24 13:08:33 2022] GET /identity/v3/endpoints?service_id=a45b06a84ee14139aeb97c5a0c3a47af&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:33.197601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:33.206222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:33.210603 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01479f4-0e3e-40a7-a039-b8eb60ab6ba5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:33.251900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af22a70-7fd5-482d-a308-bc9877e7e8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:33.253014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22a70-7fd5-482d-a308-bc9877e7e8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.253393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22a70-7fd5-482d-a308-bc9877e7e8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.253757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22a70-7fd5-482d-a308-bc9877e7e8a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.311379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:33.323714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2e181-0136-4669-bef5-6a0d4ee0dfea None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:33.330133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af22a70-7fd5-482d-a308-bc9877e7e8a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:33.331745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af22a70-7fd5-482d-a308-bc9877e7e8a7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:33.366587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca77bf-bd86-470a-bc67-7b0281c83596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:33.367854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca77bf-bd86-470a-bc67-7b0281c83596 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.368135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca77bf-bd86-470a-bc67-7b0281c83596 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.368438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca77bf-bd86-470a-bc67-7b0281c83596 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.369704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca77bf-bd86-470a-bc67-7b0281c83596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:33.371071 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca77bf-bd86-470a-bc67-7b0281c83596 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:33.380375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 354/701] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:08:33 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:08:33.452098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:33.457221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 348/702] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:33 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 24 13:08:33.462926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca77bf-bd86-470a-bc67-7b0281c83596 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:33.586572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca77bf-bd86-470a-bc67-7b0281c83596 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:33.622499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582d527-2ac2-42df-a0b9-7b1d9efe23bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:33.623131 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582d527-2ac2-42df-a0b9-7b1d9efe23bc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.623411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582d527-2ac2-42df-a0b9-7b1d9efe23bc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.623682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582d527-2ac2-42df-a0b9-7b1d9efe23bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.634379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582d527-2ac2-42df-a0b9-7b1d9efe23bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:33.635836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582d527-2ac2-42df-a0b9-7b1d9efe23bc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:33.651205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22201c-d446-43d8-8854-68e53d37811e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-d582d527-2ac2-42df-a0b9-7b1d9efe23bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:33.651507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:33.652491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:33.653488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:33.653488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:33.653488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:33.653672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22201c-d446-43d8-8854-68e53d37811e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.653672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22201c-d446-43d8-8854-68e53d37811e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.653798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 355/703] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:33.653867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 349/704] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:33.658164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22201c-d446-43d8-8854-68e53d37811e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:33.658448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d8ce-3284-4c87-be3c-94355e315614 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.658734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d8ce-3284-4c87-be3c-94355e315614 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.659008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d8ce-3284-4c87-be3c-94355e315614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.676254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:33.682894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c135d8ce-3284-4c87-be3c-94355e315614 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:33.684514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c135d8ce-3284-4c87-be3c-94355e315614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:33.766178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 356/705] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:33.780690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22201c-d446-43d8-8854-68e53d37811e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:33.815767 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:33.816696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.817185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.817648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.818462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:33.818942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:33.820516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:33.832512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 350/706] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:33.844294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d8ce-3284-4c87-be3c-94355e315614 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:33.947119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d8ce-3284-4c87-be3c-94355e315614 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:33.982359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:33.982985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:33.983271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:33.983544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:33.986239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:33.991176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:33.993380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:34.004720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:34.005679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:34.006623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 357/707] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:34.012615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:34.065977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:34.074031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:34.122746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851544e-5856-4576-8092-2e7250a2822a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:34.163349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:34.164302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:34.164762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:34.165211 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:34.166084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:34.166611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:34.168227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:34.182370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 351/708] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:34.190381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:34.202119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32b71d-00e8-4317-9bd0-1ce0beeb40a9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:34.248187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:34.248856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:34.249127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:34.249394 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:34.250106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:34.251867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:34.264859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:34.265924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:34.267366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 358/709] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:34.271884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:34.313359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0b21-6c49-435a-b73c-36ac0a0d7d29 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:34.336297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:34.345203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:34.352260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:34.353260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:34.353713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:34.354151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:34.360639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] RBAC: Adding query filter params (role.id=8d9d7fa07c94446e82a45b539babe09d, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=038f2ef649844de297237a0c1ec652aa) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:34.360639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:34.360639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:34.371202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 352/710] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:34 2022] GET /identity/v3/role_assignments?user.id=038f2ef649844de297237a0c1ec652aa&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=8d9d7fa07c94446e82a45b539babe09d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:34.392989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb5aad-fa9b-43c7-9338-e0a84418ddaf None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:34.431408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:34.432028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:34.432328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:34.432603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:34.433235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:34.433537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:34.434906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:34.440836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 359/711] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:34.446977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:34.558529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692463c-7e38-4ae1-90b0-e255549f5384 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:34.594654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d83cb2-3ea5-4000-acd2-5aa0e22ac527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:34.595238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83cb2-3ea5-4000-acd2-5aa0e22ac527 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:34.595512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83cb2-3ea5-4000-acd2-5aa0e22ac527 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:34.595777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83cb2-3ea5-4000-acd2-5aa0e22ac527 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:34.596473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83cb2-3ea5-4000-acd2-5aa0e22ac527 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:34.596772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83cb2-3ea5-4000-acd2-5aa0e22ac527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:34.598128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83cb2-3ea5-4000-acd2-5aa0e22ac527 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:34.603511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 353/712] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:34.608894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:34.712224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4574a37-ce21-4c80-9abd-7f46f620f63e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:34.744690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:34.748970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97189c-c804-4bbe-be6e-fa532c4c23a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:34.749626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97189c-c804-4bbe-be6e-fa532c4c23a6 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:34.749906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97189c-c804-4bbe-be6e-fa532c4c23a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:34.750173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97189c-c804-4bbe-be6e-fa532c4c23a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:34.750841 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97189c-c804-4bbe-be6e-fa532c4c23a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:34.752304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97189c-c804-4bbe-be6e-fa532c4c23a6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:34.822493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 360/713] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:34 2022] POST /identity/v3/endpoints => generated 349 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:35.183445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293fa4-07e6-4d13-bd73-2254e59f7f7b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:35.183901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293fa4-07e6-4d13-bd73-2254e59f7f7b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:35.184293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293fa4-07e6-4d13-bd73-2254e59f7f7b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:35.185980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 354/714] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:35.191342 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d5887-6547-4ca0-8024-c437f0f5c079 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:35.191794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d5887-6547-4ca0-8024-c437f0f5c079 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:35.192166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d5887-6547-4ca0-8024-c437f0f5c079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:35.212575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6c9d5887-6547-4ca0-8024-c437f0f5c079 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:35.213751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6c9d5887-6547-4ca0-8024-c437f0f5c079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:35.291030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 361/715] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:35.561684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0293fa4-07e6-4d13-bd73-2254e59f7f7b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:35.615325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:35.636259 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616d655-eba0-4a76-a194-b642768fe3b7 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:35.636792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616d655-eba0-4a76-a194-b642768fe3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:35.637005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616d655-eba0-4a76-a194-b642768fe3b7 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:35.638482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 362/716] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:35.643749 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788cc56-17a9-4134-b73d-6e54f8cf435d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:35.644064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788cc56-17a9-4134-b73d-6e54f8cf435d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:35.644318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788cc56-17a9-4134-b73d-6e54f8cf435d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:35.664623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5788cc56-17a9-4134-b73d-6e54f8cf435d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:35.666129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5788cc56-17a9-4134-b73d-6e54f8cf435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:35.679155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0293fa4-07e6-4d13-bd73-2254e59f7f7b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:35.729810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:35.730453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:35.730729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:35.730999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:35.731601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:35.733586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:35.745407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:35.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:35.748083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 355/718] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:35.749269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 363/718] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:35.752145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:35.852235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28f2ab-05e8-4b58-a2d2-8045ab0f966b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:35.865266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:35.886679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:35.887271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:35.887547 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:35.887842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:35.888515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:35.888817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:35.890165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:35.894995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 356/719] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:35.901050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788cc56-17a9-4134-b73d-6e54f8cf435d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:35.937946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:36.006284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788cc56-17a9-4134-b73d-6e54f8cf435d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:36.017259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:36.051332 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:36.051937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:36.052355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:36.052647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:36.053398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:36.053700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:36.055601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:36.061537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 364/720] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:36.136673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb3164-37ed-40ef-a07b-e2ae147197d3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:36.178337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:36.179556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:36.179841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:36.180119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:36.180738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:36.182129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:36.192170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:36.193275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:36.194134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 357/721] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:36.201695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:36.305152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:36.307898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb394c-4be4-41d1-aa45-6ae5b49dc4a8 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:36.342902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:36.343546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:36.343823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:36.344137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:36.344852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:36.345370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:36.347010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:36.352911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 365/722] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:36.358331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:36.404711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:36.472971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f83a1c-0ea7-44b4-b85c-0805f1c2fb8e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:36.627329 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9965938-404a-4023-8eb1-a9918c426735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:36.628778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965938-404a-4023-8eb1-a9918c426735 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:36.629263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965938-404a-4023-8eb1-a9918c426735 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:36.629737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9965938-404a-4023-8eb1-a9918c426735 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:36.630560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9965938-404a-4023-8eb1-a9918c426735 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:36.631093 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9965938-404a-4023-8eb1-a9918c426735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:36.632715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9965938-404a-4023-8eb1-a9918c426735 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:36.638485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 358/723] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Oct 24 13:08:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:36.644538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:36.711870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde059-87db-48f5-86e0-710c7f289717 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:36.712060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde059-87db-48f5-86e0-710c7f289717 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:36.712367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde059-87db-48f5-86e0-710c7f289717 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:36.715068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 359/724] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:36.719334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f2550-afbf-45c0-9d66-fa11acf848c1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:36.719622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f2550-afbf-45c0-9d66-fa11acf848c1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:36.719896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f2550-afbf-45c0-9d66-fa11acf848c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:36.738987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b46f2550-afbf-45c0-9d66-fa11acf848c1 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:36.740319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b46f2550-afbf-45c0-9d66-fa11acf848c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:36.747667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e19be5-ad04-46e3-b0b4-85f5eb9bd677 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:36.789863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074240e5-a162-4f61-9c1a-56b71b2c576f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:36.790469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074240e5-a162-4f61-9c1a-56b71b2c576f None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:36.790750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074240e5-a162-4f61-9c1a-56b71b2c576f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:36.791066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074240e5-a162-4f61-9c1a-56b71b2c576f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:36.791736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074240e5-a162-4f61-9c1a-56b71b2c576f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:36.793137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074240e5-a162-4f61-9c1a-56b71b2c576f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:36.821785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 360/725] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:36.871267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 366/726] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:37.016460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb175-af6c-41d4-811d-8779fe4b086f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:37.016645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb175-af6c-41d4-811d-8779fe4b086f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:37.017103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb175-af6c-41d4-811d-8779fe4b086f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:37.018928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 361/727] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:37.025816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c1a98-0e2d-460f-af42-e6b1eabfa57b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:37.026109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c1a98-0e2d-460f-af42-e6b1eabfa57b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:37.026523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c1a98-0e2d-460f-af42-e6b1eabfa57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:37.046885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7c7c1a98-0e2d-460f-af42-e6b1eabfa57b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:37.047548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7c7c1a98-0e2d-460f-af42-e6b1eabfa57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:37.062881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:37.100131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddb175-af6c-41d4-811d-8779fe4b086f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:37.141481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 367/728] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:37.151979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:37.220643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddb175-af6c-41d4-811d-8779fe4b086f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:37.256114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:37.256770 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:37.257583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:37.257913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] PATH_INFO: `/v3/services/image` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:37.258488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:37.259828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:37.268889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:37.270065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:37.270933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 362/729] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Oct 24 13:08:37 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:37.276478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c1a98-0e2d-460f-af42-e6b1eabfa57b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:37.398226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c1a98-0e2d-460f-af42-e6b1eabfa57b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:37.425306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:37.438984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:37.439794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:37.440073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:37.440382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:37.441081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:37.441385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:37.442758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:37.449975 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 368/730] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:37.459531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:37.484652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:37.552773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:37.556780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7dbc61-9f2d-44aa-bea5-8cd776e52673 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:37.572897 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffdd80-5e79-4ed8-b656-5b51099e57b6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:37.599061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:37.600015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:37.600378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:37.601436 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:37.602222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:37.604936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:37.616494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:37.617530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:37.618400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:37.619178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:37.619178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:37.619178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:37.619178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:37.619491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] RBAC: Adding query filter params (type=image) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:37.619939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:37.621464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:37.621681 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 363/731] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:37.627014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:37.632076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 369/732] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:37.752068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952396d3-b9c4-4957-86b6-ef1c2301f2d1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:37.792405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:37.793394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:37.793910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:37.794523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:37.795388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:37.795907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:37.797545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:37.803999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 364/733] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:37.810158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:37.889060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:37.916390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6660d1-0d9b-4738-b2f2-7484bcfbbaac None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:37.953213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:37.954240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:37.954712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:37.955187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:37.956014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:37.956535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:37.958117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:37.964564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 370/734] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:37.970956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:38.001710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:38.076773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdbf3d-cec7-4678-a054-92c6c329a4d3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:38.112790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bfa1a-09bc-4877-9316-04ddfafed6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:38.113385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bfa1a-09bc-4877-9316-04ddfafed6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:38.113657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bfa1a-09bc-4877-9316-04ddfafed6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:38.113924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bfa1a-09bc-4877-9316-04ddfafed6e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:38.114613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bfa1a-09bc-4877-9316-04ddfafed6e9 None admin] RBAC: Adding query filter params (interface=admin, service_id=60e16e9ffb804708ba7e4697d1a38a25, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:38.114916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bfa1a-09bc-4877-9316-04ddfafed6e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:38.116890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bfa1a-09bc-4877-9316-04ddfafed6e9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:38.122270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 365/735] 10.222.0.48 () {56 vars in 1249 bytes} [Mon Oct 24 13:08:37 2022] GET /identity/v3/endpoints?service_id=60e16e9ffb804708ba7e4697d1a38a25&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:38.128009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:38.234135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce590a-7bdb-44f0-9959-0d7c121d4b06 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:38.275012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f8e53-b0fe-484c-85e2-a8104b49015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:38.275618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f8e53-b0fe-484c-85e2-a8104b49015c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:38.275896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f8e53-b0fe-484c-85e2-a8104b49015c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:38.276169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f8e53-b0fe-484c-85e2-a8104b49015c None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:38.276803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f8e53-b0fe-484c-85e2-a8104b49015c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:38.278225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f8e53-b0fe-484c-85e2-a8104b49015c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:38.284890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 371/736] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:08:38 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:08:38.365069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8224-d171-43e1-82b7-a5a6eb21179f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:38.365688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8224-d171-43e1-82b7-a5a6eb21179f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:38.366167 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda8224-d171-43e1-82b7-a5a6eb21179f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:38.368331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 366/737] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:38.374375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1aaf-22da-423d-98c7-510f5625ab7c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:38.374996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1aaf-22da-423d-98c7-510f5625ab7c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:38.375476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a1aaf-22da-423d-98c7-510f5625ab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:38.382400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:38.394833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ae5a1aaf-22da-423d-98c7-510f5625ab7c None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:38.396046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ae5a1aaf-22da-423d-98c7-510f5625ab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:38.479081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 372/738] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:38.749374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda8224-d171-43e1-82b7-a5a6eb21179f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:38.843933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:38.861279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda8224-d171-43e1-82b7-a5a6eb21179f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:38.897201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8055c14-b5cd-468c-b169-337873d68e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:38.897828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055c14-b5cd-468c-b169-337873d68e5a None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:38.898672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055c14-b5cd-468c-b169-337873d68e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:38.898941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055c14-b5cd-468c-b169-337873d68e5a None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:38.899591 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8055c14-b5cd-468c-b169-337873d68e5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:38.900967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8055c14-b5cd-468c-b169-337873d68e5a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:38.965327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 367/739] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:08:38 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:39.245087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a907ad7-5492-4a0b-b526-4960f342989f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:39.246073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a907ad7-5492-4a0b-b526-4960f342989f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:39.246555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a907ad7-5492-4a0b-b526-4960f342989f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:39.248371 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 373/740] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:39.253930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38820b-982d-46c5-ab83-cf8822a3b238 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:39.254446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38820b-982d-46c5-ab83-cf8822a3b238 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:39.254944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38820b-982d-46c5-ab83-cf8822a3b238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:39.274582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9a38820b-982d-46c5-ab83-cf8822a3b238 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:39.276101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9a38820b-982d-46c5-ab83-cf8822a3b238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:39.352148 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 368/741] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:39.628182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a907ad7-5492-4a0b-b526-4960f342989f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:39.739003 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a907ad7-5492-4a0b-b526-4960f342989f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:39.772228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:39.778695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:39.779330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:39.779606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:39.779870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:39.780719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:39.782282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:39.785909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING keystone.server.flask.common [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 24 13:08:39.837460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.sql.core [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:08:39.837460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:08:39.837460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [parameters: {'id': '9cbf87c7026b48798d994326fcedacb2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 24 13:08:39.837460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-4c68fd89-e9b3-4511-97ea-df700f9c01af None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9cbf87c7026b48798d994326fcedacb2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9cbf87c7026b48798d994326fcedacb2. Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:08:39.855696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 13:08:39.856696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application yield resource Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise self.value Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application session.commit() Oct 24 13:08:39.857704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:08:39.858640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:08:39.859714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:08:39.860596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [parameters: {'id': '9cbf87c7026b48798d994326fcedacb2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:39.861507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:39.862338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9cbf87c7026b48798d994326fcedacb2. Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:39.863325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 374/742] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Oct 24 13:08:39 2022] POST /identity/v3/projects => generated 262 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:39.873918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38820b-982d-46c5-ab83-cf8822a3b238 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:39.889674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6a72-a4bf-4830-9915-a0d3fceb52f5 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:39.890235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6a72-a4bf-4830-9915-a0d3fceb52f5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:39.890688 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6a72-a4bf-4830-9915-a0d3fceb52f5 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:39.895340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 375/743] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:39.900728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28def26-e67b-4710-af7f-013a3b3ebc7f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:39.901237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28def26-e67b-4710-af7f-013a3b3ebc7f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:39.901715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28def26-e67b-4710-af7f-013a3b3ebc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:39.923215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c28def26-e67b-4710-af7f-013a3b3ebc7f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:39.925301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c28def26-e67b-4710-af7f-013a3b3ebc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:39.962584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:39.990561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38820b-982d-46c5-ab83-cf8822a3b238 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:40.008401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 376/744] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:40.035286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:40.036177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:40.036779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:40.037196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:40.044820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:40.046862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:40.058552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:40.059595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:40.060711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:40.060711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:40.060711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:40.060711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:40.060711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:40.060711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:40.060711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:40.060711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:40.062780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 369/745] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:40.072333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28def26-e67b-4710-af7f-013a3b3ebc7f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:40.182923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28def26-e67b-4710-af7f-013a3b3ebc7f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:40.200992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:40.219131 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ef94f-2c64-4607-94bd-0b669d0c9b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:40.219733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ef94f-2c64-4607-94bd-0b669d0c9b34 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:40.220007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ef94f-2c64-4607-94bd-0b669d0c9b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:40.220286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ef94f-2c64-4607-94bd-0b669d0c9b34 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:40.220936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ef94f-2c64-4607-94bd-0b669d0c9b34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:40.221233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ef94f-2c64-4607-94bd-0b669d0c9b34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:40.222813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ef94f-2c64-4607-94bd-0b669d0c9b34 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:40.240318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 377/746] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:40.283855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:40.402131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5123a8-42bb-4540-b4a5-a445098fdabd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:40.438074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1da2b4-f9ec-4a26-bb9b-f9afd252285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:40.438606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da2b4-f9ec-4a26-bb9b-f9afd252285d None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:40.439114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da2b4-f9ec-4a26-bb9b-f9afd252285d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:40.439398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da2b4-f9ec-4a26-bb9b-f9afd252285d None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:40.440044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1da2b4-f9ec-4a26-bb9b-f9afd252285d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:40.441426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1da2b4-f9ec-4a26-bb9b-f9afd252285d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:40.446717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:40.535739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 370/747] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:08:40 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:40.575263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156a3c2-c54e-4bc0-81fc-7c77678c45f3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:40.576254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156a3c2-c54e-4bc0-81fc-7c77678c45f3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:40.576809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156a3c2-c54e-4bc0-81fc-7c77678c45f3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:40.578773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 378/748] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:40.584855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7deed6-1261-4a6e-b65c-c6fa3cb03254 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:40.585517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7deed6-1261-4a6e-b65c-c6fa3cb03254 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:40.586007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7deed6-1261-4a6e-b65c-c6fa3cb03254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:40.606170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9c7deed6-1261-4a6e-b65c-c6fa3cb03254 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:40.607736 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9c7deed6-1261-4a6e-b65c-c6fa3cb03254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:40.695892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 371/749] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:40 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:40.961235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156a3c2-c54e-4bc0-81fc-7c77678c45f3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:41.068143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156a3c2-c54e-4bc0-81fc-7c77678c45f3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:41.111830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b938e-bda5-4664-a994-1102adfcd9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:41.112489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b938e-bda5-4664-a994-1102adfcd9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:41.112769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b938e-bda5-4664-a994-1102adfcd9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:41.113038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b938e-bda5-4664-a994-1102adfcd9d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:41.113615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b938e-bda5-4664-a994-1102adfcd9d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:41.114981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b938e-bda5-4664-a994-1102adfcd9d6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a44b938e-bda5-4664-a994-1102adfcd9d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:41.123960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:41.124999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:41.126090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 379/750] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Oct 24 13:08:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:41.133692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7deed6-1261-4a6e-b65c-c6fa3cb03254 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:41.245869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7deed6-1261-4a6e-b65c-c6fa3cb03254 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:41.269769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:41.272162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218de190-c0e1-426e-89e8-fba62619cb2d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:41.273265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218de190-c0e1-426e-89e8-fba62619cb2d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:41.273747 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218de190-c0e1-426e-89e8-fba62619cb2d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:41.279297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 380/751] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:41.280716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f09f9-6003-4ab1-a33d-42737dcce6a3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:41.280999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f09f9-6003-4ab1-a33d-42737dcce6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:41.281270 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f09f9-6003-4ab1-a33d-42737dcce6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:41.286485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:41.287333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:41.295171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:41.295171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:41.295171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:41.295171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:41.295171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:41.300422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 372/752] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:41.301671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0a0f09f9-6003-4ab1-a33d-42737dcce6a3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:41.302853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0a0f09f9-6003-4ab1-a33d-42737dcce6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:41.305222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:41.345493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:41.386063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 381/753] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:41 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:41.416086 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed6702-67d0-4ede-a6da-d0db119d1b91 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:41.451018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:41.451890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:41.452341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:41.452760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:41.453533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:41.453999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:41.455563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:41.462061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 373/754] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Oct 24 13:08:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:41.469704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f09f9-6003-4ab1-a33d-42737dcce6a3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:41.573591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f09f9-6003-4ab1-a33d-42737dcce6a3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:41.609452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396e2b5-d022-4f7b-9db3-8727ec12ce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:41.610939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396e2b5-d022-4f7b-9db3-8727ec12ce87 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:41.611424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396e2b5-d022-4f7b-9db3-8727ec12ce87 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:41.611900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396e2b5-d022-4f7b-9db3-8727ec12ce87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:41.612782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396e2b5-d022-4f7b-9db3-8727ec12ce87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:41.614361 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396e2b5-d022-4f7b-9db3-8727ec12ce87 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:41.679790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:41.700583 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 382/755] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:41.735683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:41.791237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5874ef1-e676-4f79-9fec-39da0485c211 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:41.835469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:41.836027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:41.836334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:41.836609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:41.837190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:41.838628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:41.848412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:41.849501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:41.851170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 374/756] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:41.855936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396e2b5-d022-4f7b-9db3-8727ec12ce87 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:41.894150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:41.966153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396e2b5-d022-4f7b-9db3-8727ec12ce87 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:42.001230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d729-5345-4284-a176-ec2581559762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:42.001833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d729-5345-4284-a176-ec2581559762 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:42.002113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d729-5345-4284-a176-ec2581559762 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:42.002382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d729-5345-4284-a176-ec2581559762 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:42.003009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189d729-5345-4284-a176-ec2581559762 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:42.003308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189d729-5345-4284-a176-ec2581559762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:42.004771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189d729-5345-4284-a176-ec2581559762 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:42.011592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 383/757] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:42.017316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:42.121710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b05e8-1357-4577-91a9-248ff2a5a7cf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:42.133654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:42.157313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:42.163688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:42.163688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:42.163688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:42.163688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:42.163688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:42.163688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:42.166729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 375/758] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:42.172555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d729-5345-4284-a176-ec2581559762 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:42.274174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:42.281199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d729-5345-4284-a176-ec2581559762 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:42.319996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f0dda-bad2-474b-aaa5-934e8f4ae507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:42.320642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f0dda-bad2-474b-aaa5-934e8f4ae507 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:42.320977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f0dda-bad2-474b-aaa5-934e8f4ae507 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:42.321255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f0dda-bad2-474b-aaa5-934e8f4ae507 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:42.322057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f0dda-bad2-474b-aaa5-934e8f4ae507 None admin] RBAC: Adding query filter params (interface=public, service_id=8c5701695b7c4936b4c0c4c98f35cc5b, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:42.322299 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f0dda-bad2-474b-aaa5-934e8f4ae507 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:42.324059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f0dda-bad2-474b-aaa5-934e8f4ae507 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:42.329955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 384/759] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Oct 24 13:08:42 2022] GET /identity/v3/endpoints?service_id=8c5701695b7c4936b4c0c4c98f35cc5b&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:42.335669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:42.443984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee24f0-8ec7-4b8d-82ae-27c6d88d128f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:42.483081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe57f5f-aba5-4133-9f46-fe48c8eea037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:42.483690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe57f5f-aba5-4133-9f46-fe48c8eea037 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:42.484027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe57f5f-aba5-4133-9f46-fe48c8eea037 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:42.484323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe57f5f-aba5-4133-9f46-fe48c8eea037 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:42.484931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe57f5f-aba5-4133-9f46-fe48c8eea037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:42.491912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe57f5f-aba5-4133-9f46-fe48c8eea037 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:42.494066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 376/760] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:08:42 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:08:42.561456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd922-6cae-4c17-8881-6e8617d5a6f7 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:42.561775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd922-6cae-4c17-8881-6e8617d5a6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:42.562042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd922-6cae-4c17-8881-6e8617d5a6f7 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:42.563845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 385/761] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:42.569313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0eddb-66f2-4eec-b31c-021940663be9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:42.569600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0eddb-66f2-4eec-b31c-021940663be9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:42.569876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0eddb-66f2-4eec-b31c-021940663be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:42.589737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-12a0eddb-66f2-4eec-b31c-021940663be9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:42.590940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-12a0eddb-66f2-4eec-b31c-021940663be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:42.624804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:42.674970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 377/762] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:42 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:42.759053 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e445cd-e397-4a92-8631-be93e685d134 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:42.759565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e445cd-e397-4a92-8631-be93e685d134 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:42.759565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e445cd-e397-4a92-8631-be93e685d134 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:42.761107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 386/763] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:42.766244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d3bc-0a3c-469d-83e3-65795c8bfa44 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:42.766542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d3bc-0a3c-469d-83e3-65795c8bfa44 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:42.766816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473d3bc-0a3c-469d-83e3-65795c8bfa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:42.785886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e473d3bc-0a3c-469d-83e3-65795c8bfa44 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:42.786999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e473d3bc-0a3c-469d-83e3-65795c8bfa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:42.854443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 378/764] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:42 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:42.953292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e445cd-e397-4a92-8631-be93e685d134 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:43.060607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e445cd-e397-4a92-8631-be93e685d134 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:43.108390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:43.109248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:43.109689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:43.110117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:43.110909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:43.112534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:43.121239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473d3bc-0a3c-469d-83e3-65795c8bfa44 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:43.169037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.sql.core [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:08:43.169037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 24 13:08:43.169037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [parameters: {'user_id': 'ad1107fb1bf04783be4b526f1d3483ee', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 24 13:08:43.169037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Oct 24 13:08:43.176062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:08:43.177292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application yield resource Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise self.value Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application session.commit() Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:08:43.178293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:08:43.179231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:08:43.180154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:08:43.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ad1107fb1bf04783be4b526f1d3483ee', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:43.181947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 24 13:08:43.182772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 24 13:08:43.183724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 24 13:08:43.183724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 24 13:08:43.183724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:08:43.183724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:08:43.183724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 24 13:08:43.183724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:43.183724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 387/765] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Oct 24 13:08:42 2022] POST /identity/v3/users => generated 160 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:43.186237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:43.221926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:43.240001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:43.244146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473d3bc-0a3c-469d-83e3-65795c8bfa44 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:43.282576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:43.283208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:43.283790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:43.284071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:43.284685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:43.286951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:43.298648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:43.301194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:43.302060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 379/766] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Oct 24 13:08:43 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:43.308322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:43.331648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca168c0-b31d-447e-a63a-58bc261f12a0 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:43.375130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:43.375998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:43.376456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:43.376900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:43.385463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:43.387275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:43.399907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:43.401087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:08:43.402203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:43.404296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 388/767] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:43.416326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:43.416478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95dd23-4888-4e49-be7c-407555f12af8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:43.460966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:43.461894 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:43.462315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:43.462730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:43.463516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] RBAC: Adding query filter params (name=image) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:43.463982 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:43.466746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:43.475772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 380/768] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:43.481254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:43.519429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:43.525938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:43.547164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66f53a-647a-4e33-8270-4531fc3c768c None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:43.595743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:43.596613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:43.597031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:43.597441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:43.598312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:43.598773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:43.600355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:43.614884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ac8a-5309-4b76-8abf-cefd3534d066 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:43.644383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 389/769] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Oct 24 13:08:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:43.654596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:43.655219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:43.655496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:43.655771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:43.656445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:43.656751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:43.658141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:43.666126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 381/770] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:43.672154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:43.788598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ed7cf-97b9-4aa6-820e-f94a89d955e7 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:43.828381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52773d97-e4cd-4f7c-8b75-e2a81fdc1bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:43.829364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52773d97-e4cd-4f7c-8b75-e2a81fdc1bcf None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:43.829848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52773d97-e4cd-4f7c-8b75-e2a81fdc1bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:43.830325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52773d97-e4cd-4f7c-8b75-e2a81fdc1bcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:43.831294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52773d97-e4cd-4f7c-8b75-e2a81fdc1bcf None admin] RBAC: Adding query filter params (interface=public, service_id=98a8d494b4ba4085abdd8a3d5e6d0485, region_id=RegionOne) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:43.831796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52773d97-e4cd-4f7c-8b75-e2a81fdc1bcf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:43.833391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52773d97-e4cd-4f7c-8b75-e2a81fdc1bcf None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:43.839449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 390/771] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Oct 24 13:08:43 2022] GET /identity/v3/endpoints?service_id=98a8d494b4ba4085abdd8a3d5e6d0485&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:43.845601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:43.947846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:43.959589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dda72-1a45-4ccc-bde9-858c7e762be7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:43.998123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ee360-67f0-45ab-821b-e8ab609c7e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:43.999081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ee360-67f0-45ab-821b-e8ab609c7e44 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:43.999574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ee360-67f0-45ab-821b-e8ab609c7e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:44.000057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ee360-67f0-45ab-821b-e8ab609c7e44 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:44.000861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ee360-67f0-45ab-821b-e8ab609c7e44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:44.002408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ee360-67f0-45ab-821b-e8ab609c7e44 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:44.012041 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 382/772] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:08:43 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:08:44.072284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8231c7f-519d-4be9-811b-d31437ed4958 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:44.072613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8231c7f-519d-4be9-811b-d31437ed4958 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:44.072852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8231c7f-519d-4be9-811b-d31437ed4958 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:44.078254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 391/773] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:44.080023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e090a9e-f1e2-47bc-aa36-1d10d5f92f28 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:44.080355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e090a9e-f1e2-47bc-aa36-1d10d5f92f28 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:44.080800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e090a9e-f1e2-47bc-aa36-1d10d5f92f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:44.102798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8e090a9e-f1e2-47bc-aa36-1d10d5f92f28 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:44.103998 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e090a9e-f1e2-47bc-aa36-1d10d5f92f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:44.181487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 383/774] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:44 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:44.441620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8231c7f-519d-4be9-811b-d31437ed4958 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:44.446120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:44.548363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8231c7f-519d-4be9-811b-d31437ed4958 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:44.583259 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa4bd7-b518-4bd0-9669-c9cffa72fd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:44.583883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa4bd7-b518-4bd0-9669-c9cffa72fd7d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:44.584158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa4bd7-b518-4bd0-9669-c9cffa72fd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:44.584455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa4bd7-b518-4bd0-9669-c9cffa72fd7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:44.586101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa4bd7-b518-4bd0-9669-c9cffa72fd7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:44.587529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa4bd7-b518-4bd0-9669-c9cffa72fd7d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:44.589246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 392/775] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:44.595397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e090a9e-f1e2-47bc-aa36-1d10d5f92f28 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:44.684874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:44.701027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e090a9e-f1e2-47bc-aa36-1d10d5f92f28 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:44.735466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2cf30-f1fd-4901-ad30-b86c725d08ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:44.736073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2cf30-f1fd-4901-ad30-b86c725d08ed None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:44.736374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2cf30-f1fd-4901-ad30-b86c725d08ed None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:44.736647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2cf30-f1fd-4901-ad30-b86c725d08ed None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:44.737289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2cf30-f1fd-4901-ad30-b86c725d08ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:44.738707 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2cf30-f1fd-4901-ad30-b86c725d08ed None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:44.767299 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe964-4bc3-4e08-a18e-f782aa72a53d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:44.767719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe964-4bc3-4e08-a18e-f782aa72a53d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:44.768055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe964-4bc3-4e08-a18e-f782aa72a53d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:44.769509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 393/776] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:44.775179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215fb55-bfb3-449d-a1c5-f23cca614906 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:44.775468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215fb55-bfb3-449d-a1c5-f23cca614906 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:44.775747 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215fb55-bfb3-449d-a1c5-f23cca614906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:44.793691 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8215fb55-bfb3-449d-a1c5-f23cca614906 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:44.795209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8215fb55-bfb3-449d-a1c5-f23cca614906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:44.838721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 384/777] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:08:44 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:08:44.874118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 394/778] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:44 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:45.147805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2cf30-f1fd-4901-ad30-b86c725d08ed None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:45.248132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2cf30-f1fd-4901-ad30-b86c725d08ed None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:45.281413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:45.282040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:45.282319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:45.282594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:45.283174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:45.284558 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:45.287136 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:45.294574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:45.295569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:08:45.296452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:45.300357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 385/779] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:45.304527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215fb55-bfb3-449d-a1c5-f23cca614906 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:45.304836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:45.403068 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215fb55-bfb3-449d-a1c5-f23cca614906 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:45.438432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:45.439344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:45.439838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:45.440325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:45.441123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:45.441609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:45.443147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:45.452449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 395/780] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:45.454437 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:45.556723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c15ba-19f4-48ba-9719-f1f7aedb4aa6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:45.590639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae7eca-4b85-4ec3-a6f7-17d1ebbb1957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:45.591239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7eca-4b85-4ec3-a6f7-17d1ebbb1957 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:45.591766 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7eca-4b85-4ec3-a6f7-17d1ebbb1957 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:45.592035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7eca-4b85-4ec3-a6f7-17d1ebbb1957 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:45.592682 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae7eca-4b85-4ec3-a6f7-17d1ebbb1957 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:45.592984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae7eca-4b85-4ec3-a6f7-17d1ebbb1957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:45.594352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae7eca-4b85-4ec3-a6f7-17d1ebbb1957 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:45.599380 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 386/781] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Oct 24 13:08:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:45.605380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:45.787038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:45.822461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e621a-3ef3-4dae-ba98-47c4f2124eb2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:45.856250 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f303ee5-563c-4232-b681-11ee77244e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:45.856862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f303ee5-563c-4232-b681-11ee77244e61 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:45.857135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f303ee5-563c-4232-b681-11ee77244e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:45.857398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f303ee5-563c-4232-b681-11ee77244e61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:45.858055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f303ee5-563c-4232-b681-11ee77244e61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:45.859406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f303ee5-563c-4232-b681-11ee77244e61 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:45.964308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 396/782] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:45 2022] POST /identity/v3/endpoints => generated 334 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:46.179241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05666c53-be58-4f30-a754-f3407be5b315 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:46.179524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05666c53-be58-4f30-a754-f3407be5b315 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:46.179796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05666c53-be58-4f30-a754-f3407be5b315 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:46.181149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 387/783] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:46.186306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ca61-53a2-4b4a-8843-0c90d0cab786 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:46.186587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ca61-53a2-4b4a-8843-0c90d0cab786 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:46.186857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ca61-53a2-4b4a-8843-0c90d0cab786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:46.205638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7d25ca61-53a2-4b4a-8843-0c90d0cab786 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:46.206809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7d25ca61-53a2-4b4a-8843-0c90d0cab786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:46.280997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 397/784] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:46.544126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05666c53-be58-4f30-a754-f3407be5b315 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:46.607934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:46.647020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05666c53-be58-4f30-a754-f3407be5b315 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:46.681397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:46.682027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:46.682310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:46.682581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] PATH_INFO: `/v3/services/image` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:46.683158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:46.684544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:46.693223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:46.694240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:46.695574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 388/785] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Oct 24 13:08:46 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:46.700163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25ca61-53a2-4b4a-8843-0c90d0cab786 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:46.749732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:46.800521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25ca61-53a2-4b4a-8843-0c90d0cab786 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:46.834336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:46.835069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:46.835298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:46.835566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:46.836177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] RBAC: Adding query filter params (name=image) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:46.836498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:46.837857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:46.843100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 398/786] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:46.848688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:46.952824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f9082-6c61-4597-912c-4f2018903a1a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:46.980718 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:46.988847 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a964d-153f-491a-8747-3f6fd0cd8743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:46.989442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a964d-153f-491a-8747-3f6fd0cd8743 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:46.989715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a964d-153f-491a-8747-3f6fd0cd8743 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:46.989978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a964d-153f-491a-8747-3f6fd0cd8743 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:46.990589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a964d-153f-491a-8747-3f6fd0cd8743 None admin] RBAC: Adding query filter params (type=image) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:46.990884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a964d-153f-491a-8747-3f6fd0cd8743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:46.992910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a964d-153f-491a-8747-3f6fd0cd8743 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:46.997597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 389/787] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:47.003265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:47.054023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390998a1-3b09-4648-a6ca-c6afea1a7b7f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.054311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390998a1-3b09-4648-a6ca-c6afea1a7b7f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.054506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390998a1-3b09-4648-a6ca-c6afea1a7b7f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.055831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 390/788] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:47.060766 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66798ff-6054-45df-a677-6992643b411a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.061054 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66798ff-6054-45df-a677-6992643b411a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.061338 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66798ff-6054-45df-a677-6992643b411a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.078964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c66798ff-6054-45df-a677-6992643b411a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:47.080150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c66798ff-6054-45df-a677-6992643b411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:47.108246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3938e93-2c7a-4eca-ae0a-e2cee701168f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:47.123371 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:47.144566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8be679-4f91-4349-aa31-9a804210332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:47.145159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be679-4f91-4349-aa31-9a804210332e None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.145441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be679-4f91-4349-aa31-9a804210332e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.145712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8be679-4f91-4349-aa31-9a804210332e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.146379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8be679-4f91-4349-aa31-9a804210332e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:47.148465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8be679-4f91-4349-aa31-9a804210332e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:47.156192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 391/789] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:47.209425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 399/790] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:47.428967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66798ff-6054-45df-a677-6992643b411a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:47.529586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66798ff-6054-45df-a677-6992643b411a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:47.552430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:47.570838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4decd0-dda4-4c29-8666-9debcb77e54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:47.571461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4decd0-dda4-4c29-8666-9debcb77e54c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.571742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4decd0-dda4-4c29-8666-9debcb77e54c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.572145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4decd0-dda4-4c29-8666-9debcb77e54c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.573253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4decd0-dda4-4c29-8666-9debcb77e54c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:47.574669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4decd0-dda4-4c29-8666-9debcb77e54c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:47.576422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 392/791] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:47.582160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8be679-4f91-4349-aa31-9a804210332e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:47.682829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8be679-4f91-4349-aa31-9a804210332e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:47.718403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0f2eb-82c7-4a6b-a104-6d432a49a918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:47.719012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0f2eb-82c7-4a6b-a104-6d432a49a918 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.719291 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0f2eb-82c7-4a6b-a104-6d432a49a918 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.719562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0f2eb-82c7-4a6b-a104-6d432a49a918 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.720220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0f2eb-82c7-4a6b-a104-6d432a49a918 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:47.721688 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0f2eb-82c7-4a6b-a104-6d432a49a918 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:47.814481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 400/792] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:08:47 2022] POST /identity/v3/users => generated 285 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:48.213376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5026b-b74e-48a0-99b0-dba667d28dcd None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.213916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5026b-b74e-48a0-99b0-dba667d28dcd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.213916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5026b-b74e-48a0-99b0-dba667d28dcd None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.215390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 393/793] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:48.220573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad74932-0aa8-4e9a-b622-1646cf6e6af8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.220955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad74932-0aa8-4e9a-b622-1646cf6e6af8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.221157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad74932-0aa8-4e9a-b622-1646cf6e6af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.238915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0ad74932-0aa8-4e9a-b622-1646cf6e6af8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:48.240090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0ad74932-0aa8-4e9a-b622-1646cf6e6af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:48.257736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:48.306035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 401/794] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:48.571930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5026b-b74e-48a0-99b0-dba667d28dcd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:48.671424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5026b-b74e-48a0-99b0-dba667d28dcd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:48.705084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edbb2b-fe0d-44d5-b6c8-7f86f13e5a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:48.705755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edbb2b-fe0d-44d5-b6c8-7f86f13e5a34 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.706038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edbb2b-fe0d-44d5-b6c8-7f86f13e5a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.706309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edbb2b-fe0d-44d5-b6c8-7f86f13e5a34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.716930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edbb2b-fe0d-44d5-b6c8-7f86f13e5a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f7edbb2b-fe0d-44d5-b6c8-7f86f13e5a34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:48.721828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:48.722883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:08:48.723745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:48.731364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 394/795] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Oct 24 13:08:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:48.731566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad74932-0aa8-4e9a-b622-1646cf6e6af8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:48.827295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad74932-0aa8-4e9a-b622-1646cf6e6af8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:48.860222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:48.860836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.861113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.861381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.862030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:48.862333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:48.863678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:48.872293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 402/796] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Oct 24 13:08:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:48.877768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edbb2b-fe0d-44d5-b6c8-7f86f13e5a34 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:48.925737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:48.979613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edbb2b-fe0d-44d5-b6c8-7f86f13e5a34 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.013191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.013811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.014087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.014358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.025017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.026445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:49.041735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:49.042705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:49.043654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:49.043654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:49.043654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:49.043654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 395/797] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:49.048611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.069282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.158641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41a551-6624-4c74-acf0-9f76134cf48d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.192524 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e843d25-9d66-4085-89c1-ca49e4315f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.193125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843d25-9d66-4085-89c1-ca49e4315f29 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.193405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843d25-9d66-4085-89c1-ca49e4315f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.193678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e843d25-9d66-4085-89c1-ca49e4315f29 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.194337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e843d25-9d66-4085-89c1-ca49e4315f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:49.194640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e843d25-9d66-4085-89c1-ca49e4315f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.195991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e843d25-9d66-4085-89c1-ca49e4315f29 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.207365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 403/798] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:49.212859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.314725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.319784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd2cf5-c7bd-4eb5-89c9-4648fb1be02a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.354149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.354698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.354973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.355242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.362289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.363720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.374993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b76db-b3ad-4315-a498-bbfb13645a03 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.375474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b76db-b3ad-4315-a498-bbfb13645a03 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.375592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b76db-b3ad-4315-a498-bbfb13645a03 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:49.375670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:49.376827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:49.377785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:49.378338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 404/799] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:49.380564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 396/800] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Oct 24 13:08:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:49.382755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.382987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2b9b4-5c22-4b19-9aec-adad963cccf4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.383255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2b9b4-5c22-4b19-9aec-adad963cccf4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.383525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2b9b4-5c22-4b19-9aec-adad963cccf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.400940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-08a2b9b4-5c22-4b19-9aec-adad963cccf4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:49.402098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-08a2b9b4-5c22-4b19-9aec-adad963cccf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:49.472355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 405/801] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:49.487266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c52bc-b3f3-450d-b502-dd95c1f914f2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.522937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.523749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.524169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.524644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.525499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:49.525941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.527484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.545114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 397/802] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Oct 24 13:08:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:49.550746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2b9b4-5c22-4b19-9aec-adad963cccf4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.560408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.651529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2b9b4-5c22-4b19-9aec-adad963cccf4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.685588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.686205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.686483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.686751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.696667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.698100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:49.713033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:49.713979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:49.714891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:49.714891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:49.714891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:49.716769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 406/803] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:49.719956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.738979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.739374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.801986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.833420 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e70dfb-741e-4c54-a779-f3ac2acf513f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.846632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1e444-58de-4d7a-b65a-30d78d657422 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.870548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.871156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.871433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.871704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.872365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:49.872668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.874279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.881184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.881956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.882233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.882499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.883188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.884637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.888150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 398/804] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:49.894203 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.922894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.sql.core [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:08:49.922894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:08:49.922894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [parameters: {'id': '2f5dbbe40fec451cbca58d4915080886', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:08:49.922894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:08:49.930859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 13:08:49.931697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application yield resource Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise self.value Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application session.commit() Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:08:49.932715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:08:49.933839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:08:49.934751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application result.read() Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:08:49.935563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [parameters: {'id': '2f5dbbe40fec451cbca58d4915080886', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 13:08:49.937460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:49.938523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 407/805] 10.222.0.48 () {60 vars in 1137 bytes} [Mon Oct 24 13:08:49 2022] POST /identity/v3/domains => generated 190 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:49.939638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.987073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:50.013979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b36c5-1218-47f0-a1ba-9323bce5c816 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.046126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30637370-a7f7-4892-9d33-c85c0337e9df None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.051183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc3760-b55b-4160-8c50-7deade8ae692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.051802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc3760-b55b-4160-8c50-7deade8ae692 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.052080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc3760-b55b-4160-8c50-7deade8ae692 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.052385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc3760-b55b-4160-8c50-7deade8ae692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.058984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc3760-b55b-4160-8c50-7deade8ae692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.060442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc3760-b55b-4160-8c50-7deade8ae692 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.067766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-97cc3760-b55b-4160-8c50-7deade8ae692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:50.071402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:50.072386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:50.073367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 399/806] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:50.073874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22ae0-0eaf-4804-b19e-ad3ed3c2af7f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.074021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22ae0-0eaf-4804-b19e-ad3ed3c2af7f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.074298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22ae0-0eaf-4804-b19e-ad3ed3c2af7f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.075515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 400/807] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:50.078333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22ae0-0eaf-4804-b19e-ad3ed3c2af7f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:50.085868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31aa99-678b-4665-8024-b29af5eb22f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.086503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31aa99-678b-4665-8024-b29af5eb22f8 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.086781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31aa99-678b-4665-8024-b29af5eb22f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.087057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31aa99-678b-4665-8024-b29af5eb22f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.098016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31aa99-678b-4665-8024-b29af5eb22f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.099455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31aa99-678b-4665-8024-b29af5eb22f8 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-8f31aa99-678b-4665-8024-b29af5eb22f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:50.114378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:50.121534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:50.122469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:50.122469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:50.122469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:50.122469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 408/808] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:50.122469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cf4ab-ff6b-4ddc-9b64-c7a14b38bf98 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.122469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cf4ab-ff6b-4ddc-9b64-c7a14b38bf98 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.122469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cf4ab-ff6b-4ddc-9b64-c7a14b38bf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.134346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ca4cf4ab-ff6b-4ddc-9b64-c7a14b38bf98 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:50.136162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ca4cf4ab-ff6b-4ddc-9b64-c7a14b38bf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:50.182837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22ae0-0eaf-4804-b19e-ad3ed3c2af7f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.203343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 409/809] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:50.204150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cf4ab-ff6b-4ddc-9b64-c7a14b38bf98 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:50.219088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.219858 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.220133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.220430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.221104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:50.221403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.222856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.225721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:50.235407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 401/810] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:50.241663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:50.304397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cf4ab-ff6b-4ddc-9b64-c7a14b38bf98 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.307483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:50.338560 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.339436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.339938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.340519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.341589 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:50.342077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.343429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.344376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64796e-ff8d-4406-93a9-11aea8b0eded None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.354949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 410/811] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:50.381188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4bb0d-1ae1-4bd7-a05e-cb6ac69c8ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.382476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4bb0d-1ae1-4bd7-a05e-cb6ac69c8ebb None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.382756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4bb0d-1ae1-4bd7-a05e-cb6ac69c8ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.383029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4bb0d-1ae1-4bd7-a05e-cb6ac69c8ebb None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/5c5b4b17788944f8a0991cb62b2a7205/roles/8d9d7fa07c94446e82a45b539babe09d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.385353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4bb0d-1ae1-4bd7-a05e-cb6ac69c8ebb None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=5c5b4b17788944f8a0991cb62b2a7205, role_id=8d9d7fa07c94446e82a45b539babe09d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.386815 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4bb0d-1ae1-4bd7-a05e-cb6ac69c8ebb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.443932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 402/812] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:08:50 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/5c5b4b17788944f8a0991cb62b2a7205/roles/8d9d7fa07c94446e82a45b539babe09d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:08:50.474286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:50.609274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacc44e-fbc5-4842-981b-b097616c53c2 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.645952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b8808-2d08-4cd5-8266-f47c03725d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.646584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8808-2d08-4cd5-8266-f47c03725d97 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.646865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8808-2d08-4cd5-8266-f47c03725d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.647240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8808-2d08-4cd5-8266-f47c03725d97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.657956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b8808-2d08-4cd5-8266-f47c03725d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-651b8808-2d08-4cd5-8266-f47c03725d97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:50.663254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:50.664201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:50.665087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 411/813] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:50.670277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4bb0d-1ae1-4bd7-a05e-cb6ac69c8ebb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:50.732832 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:50.776300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4bb0d-1ae1-4bd7-a05e-cb6ac69c8ebb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.809827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.810423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.810701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.810971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.811622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:50.811924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.813296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.821162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 403/814] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:50.826857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b8808-2d08-4cd5-8266-f47c03725d97 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:50.853846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:50.925657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b8808-2d08-4cd5-8266-f47c03725d97 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.957456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466e865-8cd7-4a15-bbbe-dd40b820b482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.958071 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466e865-8cd7-4a15-bbbe-dd40b820b482 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.958346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466e865-8cd7-4a15-bbbe-dd40b820b482 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.958611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466e865-8cd7-4a15-bbbe-dd40b820b482 None admin] PATH_INFO: `/v3/users/d21dd3e22ec949c0b54a46792fd92d29` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.972075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466e865-8cd7-4a15-bbbe-dd40b820b482 None admin] RBAC: Authorizing `identity:get_user(user_id=d21dd3e22ec949c0b54a46792fd92d29)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.973528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466e865-8cd7-4a15-bbbe-dd40b820b482 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.975115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 412/815] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:08:50 2022] GET /identity/v3/users/d21dd3e22ec949c0b54a46792fd92d29 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:50.981350 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:51.076970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff2283-f154-4392-a7d9-143d5f13236e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:51.096911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:51.113994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4a335-8213-45ec-8d93-e65f1edc8e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:51.114614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4a335-8213-45ec-8d93-e65f1edc8e14 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:51.114889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4a335-8213-45ec-8d93-e65f1edc8e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:51.115158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4a335-8213-45ec-8d93-e65f1edc8e14 None admin] PATH_INFO: `/v3/projects/6cee6630719c491db283ca32c28640ae` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:51.116326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4a335-8213-45ec-8d93-e65f1edc8e14 None admin] RBAC: Authorizing `identity:get_project(project_id=6cee6630719c491db283ca32c28640ae)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:51.117740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4a335-8213-45ec-8d93-e65f1edc8e14 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:51.119338 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 404/816] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:08:50 2022] GET /identity/v3/projects/6cee6630719c491db283ca32c28640ae => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:51.125163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466e865-8cd7-4a15-bbbe-dd40b820b482 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:51.204030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:51.221860 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466e865-8cd7-4a15-bbbe-dd40b820b482 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:51.254935 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830c896-6b44-4133-a714-c594515c302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:51.255572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c896-6b44-4133-a714-c594515c302e None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:51.255846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c896-6b44-4133-a714-c594515c302e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:51.256110 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c896-6b44-4133-a714-c594515c302e None admin] PATH_INFO: `/v3/projects/6cee6630719c491db283ca32c28640ae/users/d21dd3e22ec949c0b54a46792fd92d29/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:51.258481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830c896-6b44-4133-a714-c594515c302e None admin] RBAC: Authorizing `identity:create_grant(project_id=6cee6630719c491db283ca32c28640ae, user_id=d21dd3e22ec949c0b54a46792fd92d29, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:51.259938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830c896-6b44-4133-a714-c594515c302e None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:51.305449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 413/817] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:08:51 2022] PUT /identity/v3/projects/6cee6630719c491db283ca32c28640ae/users/d21dd3e22ec949c0b54a46792fd92d29/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:08:52.597866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345fafd-6773-4315-9219-29993191ad31 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:52.598397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345fafd-6773-4315-9219-29993191ad31 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:52.598397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345fafd-6773-4315-9219-29993191ad31 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:52.599752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 405/818] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:52.605661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0b393-e40e-4fba-a37b-17cd55352bd2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:52.606068 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0b393-e40e-4fba-a37b-17cd55352bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:52.606209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0b393-e40e-4fba-a37b-17cd55352bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:52.623791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-38f0b393-e40e-4fba-a37b-17cd55352bd2 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:52.624979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-38f0b393-e40e-4fba-a37b-17cd55352bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:52.688393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 414/819] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:52.955954 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345fafd-6773-4315-9219-29993191ad31 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:53.056200 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345fafd-6773-4315-9219-29993191ad31 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:53.089678 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749d70a-9f0a-463f-884d-ca5e178a06aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:53.090303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749d70a-9f0a-463f-884d-ca5e178a06aa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.090580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749d70a-9f0a-463f-884d-ca5e178a06aa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.090850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749d70a-9f0a-463f-884d-ca5e178a06aa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.102024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749d70a-9f0a-463f-884d-ca5e178a06aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6749d70a-9f0a-463f-884d-ca5e178a06aa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:53.106842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:53.107850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:53.108755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 406/820] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Oct 24 13:08:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:53.113991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0b393-e40e-4fba-a37b-17cd55352bd2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:53.182290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:53.210682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0b393-e40e-4fba-a37b-17cd55352bd2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:53.244502 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ede09-ed08-4489-aba0-13cfba317f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:53.245097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ede09-ed08-4489-aba0-13cfba317f48 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.245430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ede09-ed08-4489-aba0-13cfba317f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.245718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ede09-ed08-4489-aba0-13cfba317f48 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.246387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ede09-ed08-4489-aba0-13cfba317f48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:53.246691 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ede09-ed08-4489-aba0-13cfba317f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:53.248047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ede09-ed08-4489-aba0-13cfba317f48 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:53.256205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 415/821] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Oct 24 13:08:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:53.262461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749d70a-9f0a-463f-884d-ca5e178a06aa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:53.311132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:53.364610 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749d70a-9f0a-463f-884d-ca5e178a06aa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:53.396595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:53.397732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.398008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.398279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.408473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:53.409897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:53.412102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a8eee-50f7-4ed8-b4be-851cc0ba3446 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.412438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a8eee-50f7-4ed8-b4be-851cc0ba3446 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.413352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a8eee-50f7-4ed8-b4be-851cc0ba3446 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.414707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 416/822] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:08:53.419843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c237-4f9a-4a10-a4ff-8393d20a72b6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.420128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c237-4f9a-4a10-a4ff-8393d20a72b6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.420436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c237-4f9a-4a10-a4ff-8393d20a72b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:53.424595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:53.425578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:53.426465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:53.426465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:53.426465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:53.426465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 407/823] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:53.431152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:53.438227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7461c237-4f9a-4a10-a4ff-8393d20a72b6 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:53.439406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7461c237-4f9a-4a10-a4ff-8393d20a72b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:53.506995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 417/824] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:08:53.526431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:53.530231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528897e-a91b-45bd-bee7-5f5d8ed62a50 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:53.562170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:53.562770 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.563050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.563323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.563957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:53.564262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:53.565644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:53.577560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 408/825] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:53.583236 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461c237-4f9a-4a10-a4ff-8393d20a72b6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:53.643708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:53.679988 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461c237-4f9a-4a10-a4ff-8393d20a72b6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:53.780022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:53.833483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:53.834106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.834381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.835187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.842673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:53.844071 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:08:53.856282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:53.857256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:08:53.858223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:53.860417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 418/826] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Oct 24 13:08:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:53.863210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:53.897485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57385be1-4884-4c53-8b9e-137c783ed3b5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:53.930878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a193eb3-d084-491e-9f19-0aa7c82d18fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:53.931493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193eb3-d084-491e-9f19-0aa7c82d18fe None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.931773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193eb3-d084-491e-9f19-0aa7c82d18fe None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.932047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193eb3-d084-491e-9f19-0aa7c82d18fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.943035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a193eb3-d084-491e-9f19-0aa7c82d18fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9a193eb3-d084-491e-9f19-0aa7c82d18fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:08:53.947738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:53.948690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:08:53.949575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 409/827] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:08:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:53.954778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a193eb3-d084-491e-9f19-0aa7c82d18fe None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:53.960706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b029fc3-16f3-4ed3-a0bf-0eef987a3a64 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:53.973992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:53.984551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:53.994900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:53.995494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:53.995773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:53.996048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:53.996798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:53.997103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:53.998519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.020251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 419/828] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Oct 24 13:08:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:54.021294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.054020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a193eb3-d084-491e-9f19-0aa7c82d18fe None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.095434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.096040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.096342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.096616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.097277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:54.097577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.098941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.107450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 410/829] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:08:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:54.113207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.118882 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26b95-d26c-4c7f-b26a-39bdd0404f64 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.153563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ee81-48af-4e2b-9668-414ee9602f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.154154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ee81-48af-4e2b-9668-414ee9602f56 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.154433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ee81-48af-4e2b-9668-414ee9602f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.154707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ee81-48af-4e2b-9668-414ee9602f56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.190959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:54.202550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:54.210778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37aa820-0f04-4a91-aa43-2f75aedea702 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.220979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6ee81-48af-4e2b-9668-414ee9602f56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.222424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6ee81-48af-4e2b-9668-414ee9602f56 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.244415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea634672-9de4-47b8-b3e7-01aa0a60236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.245033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea634672-9de4-47b8-b3e7-01aa0a60236c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.245307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea634672-9de4-47b8-b3e7-01aa0a60236c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.245580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea634672-9de4-47b8-b3e7-01aa0a60236c None admin] PATH_INFO: `/v3/users/d21dd3e22ec949c0b54a46792fd92d29` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.246697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea634672-9de4-47b8-b3e7-01aa0a60236c None admin] RBAC: Authorizing `identity:get_user(user_id=d21dd3e22ec949c0b54a46792fd92d29)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.248097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea634672-9de4-47b8-b3e7-01aa0a60236c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.249680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 411/830] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/users/d21dd3e22ec949c0b54a46792fd92d29 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:54.255412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea634672-9de4-47b8-b3e7-01aa0a60236c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.318113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 420/831] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 24 13:08:54.324100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ee81-48af-4e2b-9668-414ee9602f56 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.353325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea634672-9de4-47b8-b3e7-01aa0a60236c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.387023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefca788-add6-4e1e-b913-72f534399bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.387617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefca788-add6-4e1e-b913-72f534399bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.387893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefca788-add6-4e1e-b913-72f534399bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.388165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefca788-add6-4e1e-b913-72f534399bd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.402212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:54.406103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:54.419940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ee81-48af-4e2b-9668-414ee9602f56 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.454327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefca788-add6-4e1e-b913-72f534399bd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.454775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.455390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.455810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.456081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.456219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefca788-add6-4e1e-b913-72f534399bd4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.466719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.468239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:08:54.484066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:54.485094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:08:54.486039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:08:54.486039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:08:54.486039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:54.486039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 421/832] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:08:54.491541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.553049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 412/833] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 24 13:08:54.558974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefca788-add6-4e1e-b913-72f534399bd4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.594967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09463d1-d50f-4dd6-928c-d0a8ea5cf976 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.606648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:54.627873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.628496 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.628775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.629046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.629681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:54.629980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.631382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.632822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:54.643077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 422/834] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:54.649165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.657424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefca788-add6-4e1e-b913-72f534399bd4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.690944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1344c75-5cba-4059-8eea-f5287e836ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.691563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1344c75-5cba-4059-8eea-f5287e836ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.691840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1344c75-5cba-4059-8eea-f5287e836ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.692113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1344c75-5cba-4059-8eea-f5287e836ce0 None admin] PATH_INFO: `/v3/projects/6cee6630719c491db283ca32c28640ae` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.693243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1344c75-5cba-4059-8eea-f5287e836ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=6cee6630719c491db283ca32c28640ae)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.694685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1344c75-5cba-4059-8eea-f5287e836ce0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.696289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 413/835] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/projects/6cee6630719c491db283ca32c28640ae => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:54.702418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1344c75-5cba-4059-8eea-f5287e836ce0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.744962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08610b3-0b85-44d2-bf1f-23d7c9ce903d None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.777960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.778579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.778855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.779122 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.785604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.787021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:08:54.798446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:08:54.799336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:08:54.800253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:08:54.800253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:08:54.800253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:08:54.800253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:08:54.800253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:08:54.800253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:08:54.800253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:08:54.800253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:08:54.800744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1344c75-5cba-4059-8eea-f5287e836ce0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.801150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 423/836] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:08:54.805234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:54.817057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:54.834208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.835489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.835809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.836102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.837099 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=6cee6630719c491db283ca32c28640ae, user.id=d21dd3e22ec949c0b54a46792fd92d29) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:54.837565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.839226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.845147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 414/837] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/role_assignments?user.id=d21dd3e22ec949c0b54a46792fd92d29&scope.project.id=6cee6630719c491db283ca32c28640ae&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:54.860522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:54.907949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cecf06-d819-43fe-8b15-5d72cbd7fa91 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:54.942839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6045c-9ec7-475d-b9a2-a7c420972f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:54.943452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6045c-9ec7-475d-b9a2-a7c420972f9f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:54.943732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6045c-9ec7-475d-b9a2-a7c420972f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:54.944567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6045c-9ec7-475d-b9a2-a7c420972f9f None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:54.945260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6045c-9ec7-475d-b9a2-a7c420972f9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:54.945563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6045c-9ec7-475d-b9a2-a7c420972f9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:54.946997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6045c-9ec7-475d-b9a2-a7c420972f9f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:54.960365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 424/838] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:08:54.966967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:55.066272 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e381d2d-7b1c-4fc2-8981-d08cb035ce47 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:55.100723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055ac99-b5fc-4a7b-8b59-e75c9a75e338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:55.101273 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055ac99-b5fc-4a7b-8b59-e75c9a75e338 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:55.101550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055ac99-b5fc-4a7b-8b59-e75c9a75e338 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:55.101823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055ac99-b5fc-4a7b-8b59-e75c9a75e338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:55.102535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055ac99-b5fc-4a7b-8b59-e75c9a75e338 None admin] RBAC: Adding query filter params (role.id=8d9d7fa07c94446e82a45b539babe09d, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=5c5b4b17788944f8a0991cb62b2a7205) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:55.102842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055ac99-b5fc-4a7b-8b59-e75c9a75e338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:55.104256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055ac99-b5fc-4a7b-8b59-e75c9a75e338 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:55.109931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 415/839] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:08:54 2022] GET /identity/v3/role_assignments?user.id=5c5b4b17788944f8a0991cb62b2a7205&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=8d9d7fa07c94446e82a45b539babe09d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:56.999819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53588abc-cecf-445f-907b-0c3a44fa1bb9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:57.000434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53588abc-cecf-445f-907b-0c3a44fa1bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:57.000434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53588abc-cecf-445f-907b-0c3a44fa1bb9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:57.001727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 425/840] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:57.007027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe0686-49a5-4321-a71e-1d10f9569a2f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:57.007470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe0686-49a5-4321-a71e-1d10f9569a2f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:57.007568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe0686-49a5-4321-a71e-1d10f9569a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:57.025666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ffbe0686-49a5-4321-a71e-1d10f9569a2f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:57.026833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ffbe0686-49a5-4321-a71e-1d10f9569a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:57.065286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:57.090491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 416/841] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:08:57.359911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53588abc-cecf-445f-907b-0c3a44fa1bb9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:57.455752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53588abc-cecf-445f-907b-0c3a44fa1bb9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:57.469691 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:57.489013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8807a45-0363-473f-9fa6-f034594ee20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:57.489611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8807a45-0363-473f-9fa6-f034594ee20c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:57.489885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8807a45-0363-473f-9fa6-f034594ee20c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:57.490152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8807a45-0363-473f-9fa6-f034594ee20c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:57.492042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8807a45-0363-473f-9fa6-f034594ee20c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:57.493483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8807a45-0363-473f-9fa6-f034594ee20c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:57.494993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 426/842] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:08:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:57.500674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe0686-49a5-4321-a71e-1d10f9569a2f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:57.595864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe0686-49a5-4321-a71e-1d10f9569a2f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:57.628200 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6d78a-5959-4770-887a-a0c4207adc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:57.628758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d78a-5959-4770-887a-a0c4207adc58 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:57.629035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d78a-5959-4770-887a-a0c4207adc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:57.629307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d78a-5959-4770-887a-a0c4207adc58 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:57.629939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6d78a-5959-4770-887a-a0c4207adc58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:57.631367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6d78a-5959-4770-887a-a0c4207adc58 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:57.699547 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 417/843] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:08:57 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:59.835981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a724c2-b1c3-4799-9a11-7a14fe8380eb None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:59.836601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a724c2-b1c3-4799-9a11-7a14fe8380eb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:59.836601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a724c2-b1c3-4799-9a11-7a14fe8380eb None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:59.837874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 427/844] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:59.843225 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5160-dd4f-4663-9bc5-7da6868da799 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:59.843670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5160-dd4f-4663-9bc5-7da6868da799 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:59.843780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5160-dd4f-4663-9bc5-7da6868da799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:59.861964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8a6e5160-dd4f-4663-9bc5-7da6868da799 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:59.863130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8a6e5160-dd4f-4663-9bc5-7da6868da799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:59.887102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:59.927662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 418/845] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:08:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:00.194772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a724c2-b1c3-4799-9a11-7a14fe8380eb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:00.293666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a724c2-b1c3-4799-9a11-7a14fe8380eb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:00.326610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6d1f-1dfe-44cb-bf4e-5ed8d4b883e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:00.327215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6d1f-1dfe-44cb-bf4e-5ed8d4b883e1 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:00.327497 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6d1f-1dfe-44cb-bf4e-5ed8d4b883e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:00.327778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6d1f-1dfe-44cb-bf4e-5ed8d4b883e1 None admin] PATH_INFO: `/v3/roles/23a3d7681bf247468a0baa9bfed6f4e9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:00.329150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6d1f-1dfe-44cb-bf4e-5ed8d4b883e1 None admin] RBAC: Authorizing `identity:get_role(role_id=23a3d7681bf247468a0baa9bfed6f4e9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:00.330496 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6d1f-1dfe-44cb-bf4e-5ed8d4b883e1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:00.331789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 428/846] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:09:00 2022] GET /identity/v3/roles/23a3d7681bf247468a0baa9bfed6f4e9 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:00.337926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e5160-dd4f-4663-9bc5-7da6868da799 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:00.432332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e5160-dd4f-4663-9bc5-7da6868da799 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:00.465753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7438397-e360-4c58-8c92-11c9dd534b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:00.466613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7438397-e360-4c58-8c92-11c9dd534b2e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:00.466889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7438397-e360-4c58-8c92-11c9dd534b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:00.467163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7438397-e360-4c58-8c92-11c9dd534b2e None admin] PATH_INFO: `/v3/users/5ff81701b5ac4e47962d37f565531c98` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:00.479732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7438397-e360-4c58-8c92-11c9dd534b2e None admin] RBAC: Authorizing `identity:get_user(user_id=5ff81701b5ac4e47962d37f565531c98)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:00.481187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7438397-e360-4c58-8c92-11c9dd534b2e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:00.482742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 419/847] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:09:00 2022] GET /identity/v3/users/5ff81701b5ac4e47962d37f565531c98 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:00.488488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6d1f-1dfe-44cb-bf4e-5ed8d4b883e1 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:00.536914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:00.584692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6d1f-1dfe-44cb-bf4e-5ed8d4b883e1 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:00.617830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de0ff5-cf64-4c84-b8c7-a89d238130e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:00.618437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0ff5-cf64-4c84-b8c7-a89d238130e9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:00.618712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0ff5-cf64-4c84-b8c7-a89d238130e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:00.618976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de0ff5-cf64-4c84-b8c7-a89d238130e9 None admin] PATH_INFO: `/v3/projects/6cee6630719c491db283ca32c28640ae` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:00.620044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de0ff5-cf64-4c84-b8c7-a89d238130e9 None admin] RBAC: Authorizing `identity:get_project(project_id=6cee6630719c491db283ca32c28640ae)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:00.621585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de0ff5-cf64-4c84-b8c7-a89d238130e9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:00.623168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 429/848] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:09:00 2022] GET /identity/v3/projects/6cee6630719c491db283ca32c28640ae => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:00.629139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7438397-e360-4c58-8c92-11c9dd534b2e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:00.648144 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:00.726022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7438397-e360-4c58-8c92-11c9dd534b2e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:00.758882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af47f9d-0aa9-4c68-9699-2735b976e1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:00.759509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af47f9d-0aa9-4c68-9699-2735b976e1a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:00.759789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af47f9d-0aa9-4c68-9699-2735b976e1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:00.760061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af47f9d-0aa9-4c68-9699-2735b976e1a9 None admin] PATH_INFO: `/v3/projects/6cee6630719c491db283ca32c28640ae/users/5ff81701b5ac4e47962d37f565531c98/roles/23a3d7681bf247468a0baa9bfed6f4e9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:00.762183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af47f9d-0aa9-4c68-9699-2735b976e1a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6cee6630719c491db283ca32c28640ae, user_id=5ff81701b5ac4e47962d37f565531c98, role_id=23a3d7681bf247468a0baa9bfed6f4e9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:00.763592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af47f9d-0aa9-4c68-9699-2735b976e1a9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:00.819512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 420/849] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:09:00 2022] PUT /identity/v3/projects/6cee6630719c491db283ca32c28640ae/users/5ff81701b5ac4e47962d37f565531c98/roles/23a3d7681bf247468a0baa9bfed6f4e9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:09:02.927271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d18436-3b48-4bf2-b9e4-c22a935a1e6b None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:02.927803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d18436-3b48-4bf2-b9e4-c22a935a1e6b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:02.927803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d18436-3b48-4bf2-b9e4-c22a935a1e6b None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:02.929684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 430/850] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:02.934820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913527c-4466-422c-a962-12d334b373ac None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:02.935239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913527c-4466-422c-a962-12d334b373ac None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:02.935367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913527c-4466-422c-a962-12d334b373ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:02.953555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4913527c-4466-422c-a962-12d334b373ac None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:02.954721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4913527c-4466-422c-a962-12d334b373ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:03.018190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 421/851] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:03.286085 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d18436-3b48-4bf2-b9e4-c22a935a1e6b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:03.384801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d18436-3b48-4bf2-b9e4-c22a935a1e6b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:03.401716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:03.417664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804f7db-b568-4720-842f-e65959f7d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:03.426974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f7db-b568-4720-842f-e65959f7d7ec None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:03.426974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f7db-b568-4720-842f-e65959f7d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:03.426974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804f7db-b568-4720-842f-e65959f7d7ec None admin] PATH_INFO: `/v3/roles/23a3d7681bf247468a0baa9bfed6f4e9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:03.426974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804f7db-b568-4720-842f-e65959f7d7ec None admin] RBAC: Authorizing `identity:get_role(role_id=23a3d7681bf247468a0baa9bfed6f4e9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:03.426974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804f7db-b568-4720-842f-e65959f7d7ec None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:03.426974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 431/852] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:09:03 2022] GET /identity/v3/roles/23a3d7681bf247468a0baa9bfed6f4e9 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:03.428428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913527c-4466-422c-a962-12d334b373ac None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:03.438336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:03.525735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913527c-4466-422c-a962-12d334b373ac None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:03.558791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b11433-3aa8-4773-894d-c5ed1d96577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:03.559349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b11433-3aa8-4773-894d-c5ed1d96577f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:03.559629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b11433-3aa8-4773-894d-c5ed1d96577f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:03.559904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b11433-3aa8-4773-894d-c5ed1d96577f None admin] PATH_INFO: `/v3/users/5ff81701b5ac4e47962d37f565531c98` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:03.561075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b11433-3aa8-4773-894d-c5ed1d96577f None admin] RBAC: Authorizing `identity:get_user(user_id=5ff81701b5ac4e47962d37f565531c98)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:03.562491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b11433-3aa8-4773-894d-c5ed1d96577f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:03.564046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 422/853] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:09:03 2022] GET /identity/v3/users/5ff81701b5ac4e47962d37f565531c98 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:03.570434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804f7db-b568-4720-842f-e65959f7d7ec None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:03.665493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804f7db-b568-4720-842f-e65959f7d7ec None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:03.698275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc87b3-bd35-404c-af89-7eade653d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:03.698861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc87b3-bd35-404c-af89-7eade653d44a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:03.699141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc87b3-bd35-404c-af89-7eade653d44a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:03.699414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc87b3-bd35-404c-af89-7eade653d44a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:03.750281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:03.763221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc87b3-bd35-404c-af89-7eade653d44a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:03.764666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc87b3-bd35-404c-af89-7eade653d44a None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:03.858782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 432/854] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:09:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:09:03.864642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b11433-3aa8-4773-894d-c5ed1d96577f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:03.939064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:03.960452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b11433-3aa8-4773-894d-c5ed1d96577f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:03.993064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c161d-9e19-4d7f-ad42-e8befedbefda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:03.993666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c161d-9e19-4d7f-ad42-e8befedbefda None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:03.993943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c161d-9e19-4d7f-ad42-e8befedbefda None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:03.994210 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c161d-9e19-4d7f-ad42-e8befedbefda None admin] PATH_INFO: `/v3/projects/6cee6630719c491db283ca32c28640ae` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:03.995295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c161d-9e19-4d7f-ad42-e8befedbefda None admin] RBAC: Authorizing `identity:get_project(project_id=6cee6630719c491db283ca32c28640ae)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:03.996731 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c161d-9e19-4d7f-ad42-e8befedbefda None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:03.998305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 423/855] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:09:03 2022] GET /identity/v3/projects/6cee6630719c491db283ca32c28640ae => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:04.004434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc87b3-bd35-404c-af89-7eade653d44a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:04.084815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:04.100785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc87b3-bd35-404c-af89-7eade653d44a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:04.135356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c6bc2-b288-43be-af2c-1760a57131b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:04.135877 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6bc2-b288-43be-af2c-1760a57131b5 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:04.136151 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6bc2-b288-43be-af2c-1760a57131b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:04.136452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6bc2-b288-43be-af2c-1760a57131b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:04.137155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c6bc2-b288-43be-af2c-1760a57131b5 None admin] RBAC: Adding query filter params (role.id=23a3d7681bf247468a0baa9bfed6f4e9, scope.project.id=6cee6630719c491db283ca32c28640ae, user.id=5ff81701b5ac4e47962d37f565531c98) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:09:04.137455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c6bc2-b288-43be-af2c-1760a57131b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:04.138855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c6bc2-b288-43be-af2c-1760a57131b5 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:04.144290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 433/856] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:09:04 2022] GET /identity/v3/role_assignments?user.id=5ff81701b5ac4e47962d37f565531c98&scope.project.id=6cee6630719c491db283ca32c28640ae&role.id=23a3d7681bf247468a0baa9bfed6f4e9 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:06.250309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ea763-3572-4435-bf68-5478bac0768e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:06.250847 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ea763-3572-4435-bf68-5478bac0768e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:06.250847 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ea763-3572-4435-bf68-5478bac0768e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:06.252171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 424/857] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:06.257535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44d797-5a69-4602-ba79-946d1fbd3932 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:06.257964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44d797-5a69-4602-ba79-946d1fbd3932 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:06.258086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44d797-5a69-4602-ba79-946d1fbd3932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:06.275794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8c44d797-5a69-4602-ba79-946d1fbd3932 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:06.277028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8c44d797-5a69-4602-ba79-946d1fbd3932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:06.341033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 434/858] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:06.611237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ea763-3572-4435-bf68-5478bac0768e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:06.713235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ea763-3572-4435-bf68-5478bac0768e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:06.748522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9212989-2215-4b81-8559-ebc0230e9621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:06.749141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9212989-2215-4b81-8559-ebc0230e9621 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:06.749421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9212989-2215-4b81-8559-ebc0230e9621 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:06.749694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9212989-2215-4b81-8559-ebc0230e9621 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:06.750790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9212989-2215-4b81-8559-ebc0230e9621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:06.752206 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9212989-2215-4b81-8559-ebc0230e9621 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:06.753946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 425/859] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:09:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:06.759844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44d797-5a69-4602-ba79-946d1fbd3932 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:06.839334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:06.858763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44d797-5a69-4602-ba79-946d1fbd3932 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:06.892527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9b49a-68f5-4b53-97eb-74e0dc34c498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:06.893057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9b49a-68f5-4b53-97eb-74e0dc34c498 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:06.893335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9b49a-68f5-4b53-97eb-74e0dc34c498 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:06.893604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9b49a-68f5-4b53-97eb-74e0dc34c498 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:06.894246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9b49a-68f5-4b53-97eb-74e0dc34c498 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:06.895678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9b49a-68f5-4b53-97eb-74e0dc34c498 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:06.969093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 435/860] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:09:06 2022] POST /identity/v3/projects => generated 311 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:09:09.116741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041f352-47f4-4568-84d9-145c785da4fd None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:09.117199 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041f352-47f4-4568-84d9-145c785da4fd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:09.117313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041f352-47f4-4568-84d9-145c785da4fd None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:09.118610 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 426/861] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:09.124013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439409f-0bae-4e9d-83ab-b7722da77629 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:09.124441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439409f-0bae-4e9d-83ab-b7722da77629 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:09.124587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439409f-0bae-4e9d-83ab-b7722da77629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:09.142755 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4439409f-0bae-4e9d-83ab-b7722da77629 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:09.144012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4439409f-0bae-4e9d-83ab-b7722da77629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:09.206689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 436/862] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:09.476750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041f352-47f4-4568-84d9-145c785da4fd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:09.491993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:09.573405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041f352-47f4-4568-84d9-145c785da4fd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:09.606029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae417d-c25d-44e3-ac88-54fe21bba50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:09.606625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae417d-c25d-44e3-ac88-54fe21bba50e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:09.606900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae417d-c25d-44e3-ac88-54fe21bba50e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:09.607167 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae417d-c25d-44e3-ac88-54fe21bba50e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:09.608237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae417d-c25d-44e3-ac88-54fe21bba50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:09.609759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae417d-c25d-44e3-ac88-54fe21bba50e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:09.611259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 427/863] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:09:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:09.616918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439409f-0bae-4e9d-83ab-b7722da77629 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:09.674723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:09.712256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439409f-0bae-4e9d-83ab-b7722da77629 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:09.745324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73611471-2971-4049-888f-eec251332a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:09.745853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611471-2971-4049-888f-eec251332a82 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:09.746130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611471-2971-4049-888f-eec251332a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:09.746401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611471-2971-4049-888f-eec251332a82 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:09.747034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73611471-2971-4049-888f-eec251332a82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:09.748473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73611471-2971-4049-888f-eec251332a82 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:09.825262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 437/864] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:09:09 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:09:11.944292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc495e1-02af-4576-a4d6-d85b7a80338e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:11.944855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc495e1-02af-4576-a4d6-d85b7a80338e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:11.944855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc495e1-02af-4576-a4d6-d85b7a80338e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:11.946163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 428/865] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:11.951292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23170a-2ccb-48b7-ab03-2b364ff71989 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:11.951699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23170a-2ccb-48b7-ab03-2b364ff71989 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:11.951834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23170a-2ccb-48b7-ab03-2b364ff71989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:11.969652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-cc23170a-2ccb-48b7-ab03-2b364ff71989 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:11.970807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cc23170a-2ccb-48b7-ab03-2b364ff71989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:12.034054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 438/866] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:12.306182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc495e1-02af-4576-a4d6-d85b7a80338e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:12.386127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:12.405645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc495e1-02af-4576-a4d6-d85b7a80338e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:12.438041 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc316586-8a3d-405a-bd73-dbf06684c04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:12.438649 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc316586-8a3d-405a-bd73-dbf06684c04e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:12.438927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc316586-8a3d-405a-bd73-dbf06684c04e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:12.439199 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc316586-8a3d-405a-bd73-dbf06684c04e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:12.449601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc316586-8a3d-405a-bd73-dbf06684c04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-fc316586-8a3d-405a-bd73-dbf06684c04e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:09:12.454315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:09:12.455545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:09:12.459700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 429/867] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:09:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:09:12.461864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23170a-2ccb-48b7-ab03-2b364ff71989 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:12.509731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:12.558546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23170a-2ccb-48b7-ab03-2b364ff71989 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:12.592357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:12.592936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:12.593209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:12.593477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:12.594120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:09:12.594417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:12.595800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:12.604065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 439/868] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:09:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:12.609933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc316586-8a3d-405a-bd73-dbf06684c04e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:12.706562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc316586-8a3d-405a-bd73-dbf06684c04e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:12.738696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c99f19-52fb-43f7-9e1c-3b6b78da5827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:12.739294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c99f19-52fb-43f7-9e1c-3b6b78da5827 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:12.739574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c99f19-52fb-43f7-9e1c-3b6b78da5827 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:12.739848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c99f19-52fb-43f7-9e1c-3b6b78da5827 None admin] PATH_INFO: `/v3/users/f74530ef68a8460fb2a7123895b88e76` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:12.750176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:12.753313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c99f19-52fb-43f7-9e1c-3b6b78da5827 None admin] RBAC: Authorizing `identity:get_user(user_id=f74530ef68a8460fb2a7123895b88e76)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:12.754729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c99f19-52fb-43f7-9e1c-3b6b78da5827 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:12.756857 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 430/869] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:09:12 2022] GET /identity/v3/users/f74530ef68a8460fb2a7123895b88e76 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:12.762468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:12.857562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1592d-5f40-4dac-a715-557443b298e3 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:12.867610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:12.889610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494d2bf-ad8e-40cb-bcd9-c118a0055c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:12.890215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494d2bf-ad8e-40cb-bcd9-c118a0055c19 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:12.890487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494d2bf-ad8e-40cb-bcd9-c118a0055c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:12.890752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494d2bf-ad8e-40cb-bcd9-c118a0055c19 None admin] PATH_INFO: `/v3/projects/1afe18ad3f4a4abdb1961c55f463d3d5` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:12.891812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494d2bf-ad8e-40cb-bcd9-c118a0055c19 None admin] RBAC: Authorizing `identity:get_project(project_id=1afe18ad3f4a4abdb1961c55f463d3d5)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:12.893313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494d2bf-ad8e-40cb-bcd9-c118a0055c19 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:12.894896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 440/870] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:09:12 2022] GET /identity/v3/projects/1afe18ad3f4a4abdb1961c55f463d3d5 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:12.900998 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c99f19-52fb-43f7-9e1c-3b6b78da5827 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:12.996392 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c99f19-52fb-43f7-9e1c-3b6b78da5827 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:13.028808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323945f9-b64c-4327-9b3c-fd67c2ddca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:13.029442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323945f9-b64c-4327-9b3c-fd67c2ddca0d None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:13.029722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323945f9-b64c-4327-9b3c-fd67c2ddca0d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:13.030001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323945f9-b64c-4327-9b3c-fd67c2ddca0d None admin] PATH_INFO: `/v3/projects/1afe18ad3f4a4abdb1961c55f463d3d5/users/f74530ef68a8460fb2a7123895b88e76/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:13.031859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323945f9-b64c-4327-9b3c-fd67c2ddca0d None admin] RBAC: Authorizing `identity:create_grant(project_id=1afe18ad3f4a4abdb1961c55f463d3d5, user_id=f74530ef68a8460fb2a7123895b88e76, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:13.033289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323945f9-b64c-4327-9b3c-fd67c2ddca0d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:13.098274 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 431/871] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:09:12 2022] PUT /identity/v3/projects/1afe18ad3f4a4abdb1961c55f463d3d5/users/f74530ef68a8460fb2a7123895b88e76/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:09:15.193946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cfb94-ceb2-47c6-908b-14dc25e0e153 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:15.194458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cfb94-ceb2-47c6-908b-14dc25e0e153 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:15.194458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cfb94-ceb2-47c6-908b-14dc25e0e153 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:15.195791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 441/872] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:15.201105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e378f0-acd8-4f3c-8d9f-6d7f99d030b8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:15.201491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e378f0-acd8-4f3c-8d9f-6d7f99d030b8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:15.201649 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e378f0-acd8-4f3c-8d9f-6d7f99d030b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:15.219976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f9e378f0-acd8-4f3c-8d9f-6d7f99d030b8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:15.221161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f9e378f0-acd8-4f3c-8d9f-6d7f99d030b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:15.271962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:15.284588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 432/873] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:15.554601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cfb94-ceb2-47c6-908b-14dc25e0e153 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:15.652349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cfb94-ceb2-47c6-908b-14dc25e0e153 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:15.684663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff67d1-045b-4f1a-9e90-196aab02574a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:15.685265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff67d1-045b-4f1a-9e90-196aab02574a None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:15.685541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff67d1-045b-4f1a-9e90-196aab02574a None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:15.685811 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff67d1-045b-4f1a-9e90-196aab02574a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:15.696292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff67d1-045b-4f1a-9e90-196aab02574a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c8ff67d1-045b-4f1a-9e90-196aab02574a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:09:15.700973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:09:15.702040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:09:15.703211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 442/874] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:09:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:09:15.707868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e378f0-acd8-4f3c-8d9f-6d7f99d030b8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:15.800970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e378f0-acd8-4f3c-8d9f-6d7f99d030b8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:15.833899 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:15.834425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:15.834702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:15.834973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:15.835620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:09:15.835920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:15.837283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:15.845231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 433/875] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:09:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:15.851018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff67d1-045b-4f1a-9e90-196aab02574a None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:15.889816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:15.947750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff67d1-045b-4f1a-9e90-196aab02574a None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:15.980309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b583732-bc38-4302-b772-4b27bd2ddfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:15.980907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b583732-bc38-4302-b772-4b27bd2ddfca None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:15.981183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b583732-bc38-4302-b772-4b27bd2ddfca None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:15.981448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b583732-bc38-4302-b772-4b27bd2ddfca None admin] PATH_INFO: `/v3/users/f74530ef68a8460fb2a7123895b88e76` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:15.982522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b583732-bc38-4302-b772-4b27bd2ddfca None admin] RBAC: Authorizing `identity:get_user(user_id=f74530ef68a8460fb2a7123895b88e76)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:15.983914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b583732-bc38-4302-b772-4b27bd2ddfca None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:15.985461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 443/876] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:09:15 2022] GET /identity/v3/users/f74530ef68a8460fb2a7123895b88e76 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:15.991821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:16.039598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:16.088408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8170ef-ae1f-4fbb-8cf8-29d892df69c5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:16.121386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505f288-9274-49ad-a289-4bc015ba522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:16.121976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f288-9274-49ad-a289-4bc015ba522f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:16.122253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f288-9274-49ad-a289-4bc015ba522f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:16.122523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f288-9274-49ad-a289-4bc015ba522f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:16.186763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505f288-9274-49ad-a289-4bc015ba522f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:16.188178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505f288-9274-49ad-a289-4bc015ba522f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:16.234080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:16.282192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 434/877] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:09:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:09:16.288077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b583732-bc38-4302-b772-4b27bd2ddfca None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:16.383645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b583732-bc38-4302-b772-4b27bd2ddfca None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:16.390532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:16.415427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff532cf5-8a58-4a33-997c-1cd2eeb1bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:16.416033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff532cf5-8a58-4a33-997c-1cd2eeb1bc57 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:16.416341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff532cf5-8a58-4a33-997c-1cd2eeb1bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:16.416615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff532cf5-8a58-4a33-997c-1cd2eeb1bc57 None admin] PATH_INFO: `/v3/projects/1afe18ad3f4a4abdb1961c55f463d3d5` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:16.417706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff532cf5-8a58-4a33-997c-1cd2eeb1bc57 None admin] RBAC: Authorizing `identity:get_project(project_id=1afe18ad3f4a4abdb1961c55f463d3d5)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:16.419374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff532cf5-8a58-4a33-997c-1cd2eeb1bc57 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:16.421209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 444/878] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:09:16 2022] GET /identity/v3/projects/1afe18ad3f4a4abdb1961c55f463d3d5 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:16.426974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505f288-9274-49ad-a289-4bc015ba522f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:16.522423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505f288-9274-49ad-a289-4bc015ba522f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:16.554378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f42373-8f5b-47d9-8d14-6270a31c27d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:16.554956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f42373-8f5b-47d9-8d14-6270a31c27d9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:16.555230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f42373-8f5b-47d9-8d14-6270a31c27d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:16.555500 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f42373-8f5b-47d9-8d14-6270a31c27d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:16.556201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f42373-8f5b-47d9-8d14-6270a31c27d9 None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=1afe18ad3f4a4abdb1961c55f463d3d5, user.id=f74530ef68a8460fb2a7123895b88e76) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:09:16.556533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f42373-8f5b-47d9-8d14-6270a31c27d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:16.557926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f42373-8f5b-47d9-8d14-6270a31c27d9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:16.563332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 435/879] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:09:16 2022] GET /identity/v3/role_assignments?user.id=f74530ef68a8460fb2a7123895b88e76&scope.project.id=1afe18ad3f4a4abdb1961c55f463d3d5&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:18.677736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846e31-1111-4d38-8963-91ad4b057842 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:18.678297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846e31-1111-4d38-8963-91ad4b057842 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:18.678297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846e31-1111-4d38-8963-91ad4b057842 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:18.679601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 445/880] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:18.684711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc4ee-d4c9-4855-a825-acfac1ad32cf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:18.685093 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc4ee-d4c9-4855-a825-acfac1ad32cf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:18.685255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcc4ee-d4c9-4855-a825-acfac1ad32cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:18.702704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9dbcc4ee-d4c9-4855-a825-acfac1ad32cf None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:18.703892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9dbcc4ee-d4c9-4855-a825-acfac1ad32cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:18.713258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:18.768467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 436/881] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:19.033894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f846e31-1111-4d38-8963-91ad4b057842 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:19.131309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f846e31-1111-4d38-8963-91ad4b057842 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:19.164470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6f60b-56ce-4b31-bdf4-6ecfd67828ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:19.165055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f60b-56ce-4b31-bdf4-6ecfd67828ea None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:19.165327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f60b-56ce-4b31-bdf4-6ecfd67828ea None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:19.165593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f60b-56ce-4b31-bdf4-6ecfd67828ea None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:19.166237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f60b-56ce-4b31-bdf4-6ecfd67828ea None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:19.167585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d6f60b-56ce-4b31-bdf4-6ecfd67828ea None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:19.236257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 446/882] 10.222.0.48 () {60 vars in 1138 bytes} [Mon Oct 24 13:09:19 2022] POST /identity/v3/domains => generated 267 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:09:21.360368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da3e59-8acc-4232-a80c-a2d5acee4f7c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:21.360860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da3e59-8acc-4232-a80c-a2d5acee4f7c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:21.360860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da3e59-8acc-4232-a80c-a2d5acee4f7c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:21.362199 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 437/883] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:21.367858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e476ab-8525-456d-818e-8f147775b907 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:21.368316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e476ab-8525-456d-818e-8f147775b907 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:21.368426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e476ab-8525-456d-818e-8f147775b907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:21.386620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a5e476ab-8525-456d-818e-8f147775b907 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:21.387794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a5e476ab-8525-456d-818e-8f147775b907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:21.413885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:21.452100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 447/884] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:21.719725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da3e59-8acc-4232-a80c-a2d5acee4f7c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:21.816686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da3e59-8acc-4232-a80c-a2d5acee4f7c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:21.849326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3241a65-4fa7-4d33-a75f-8cdfc63be6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:21.849932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3241a65-4fa7-4d33-a75f-8cdfc63be6fe None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:21.850208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3241a65-4fa7-4d33-a75f-8cdfc63be6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:21.850477 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3241a65-4fa7-4d33-a75f-8cdfc63be6fe None admin] PATH_INFO: `/v3/domains/608c29cd74f447a9be70ae013f732e8e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:21.860352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3241a65-4fa7-4d33-a75f-8cdfc63be6fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=608c29cd74f447a9be70ae013f732e8e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:21.861801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3241a65-4fa7-4d33-a75f-8cdfc63be6fe None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:21.863412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 438/885] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Oct 24 13:09:21 2022] GET /identity/v3/domains/608c29cd74f447a9be70ae013f732e8e => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:21.869563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e476ab-8525-456d-818e-8f147775b907 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:21.963539 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e476ab-8525-456d-818e-8f147775b907 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:21.997915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50d926-c751-4b4a-ad17-1e698959f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:21.998453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d926-c751-4b4a-ad17-1e698959f250 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:21.998739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d926-c751-4b4a-ad17-1e698959f250 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:21.999011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50d926-c751-4b4a-ad17-1e698959f250 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:21.999659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50d926-c751-4b4a-ad17-1e698959f250 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:22.001114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50d926-c751-4b4a-ad17-1e698959f250 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:22.106251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 448/886] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:09:21 2022] POST /identity/v3/projects => generated 361 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:09:24.265592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870fd17-dea6-446e-859a-02ff9fb33394 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:24.266120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870fd17-dea6-446e-859a-02ff9fb33394 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:24.266120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870fd17-dea6-446e-859a-02ff9fb33394 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:24.267441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 439/887] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:24.272864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88743-af25-4a3e-96bb-3a9f4eb9c471 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:24.273272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88743-af25-4a3e-96bb-3a9f4eb9c471 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:24.273414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88743-af25-4a3e-96bb-3a9f4eb9c471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:24.291540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9fb88743-af25-4a3e-96bb-3a9f4eb9c471 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:24.292742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9fb88743-af25-4a3e-96bb-3a9f4eb9c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:24.303840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:24.357201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 449/888] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:24.625422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870fd17-dea6-446e-859a-02ff9fb33394 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:24.629151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:24.724612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870fd17-dea6-446e-859a-02ff9fb33394 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:24.756725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3801317-bf86-4640-a5ff-0be2cc77bdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:24.757327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3801317-bf86-4640-a5ff-0be2cc77bdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:24.757605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3801317-bf86-4640-a5ff-0be2cc77bdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:24.757875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3801317-bf86-4640-a5ff-0be2cc77bdc2 None admin] PATH_INFO: `/v3/domains/608c29cd74f447a9be70ae013f732e8e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:24.758962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3801317-bf86-4640-a5ff-0be2cc77bdc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=608c29cd74f447a9be70ae013f732e8e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:24.760390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3801317-bf86-4640-a5ff-0be2cc77bdc2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:24.761906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 440/889] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Oct 24 13:09:24 2022] GET /identity/v3/domains/608c29cd74f447a9be70ae013f732e8e => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:24.767628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb88743-af25-4a3e-96bb-3a9f4eb9c471 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:24.862360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb88743-af25-4a3e-96bb-3a9f4eb9c471 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:24.895317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a1aaf-7016-49b8-85d3-8431874125c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:24.895927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1aaf-7016-49b8-85d3-8431874125c1 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:24.896207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1aaf-7016-49b8-85d3-8431874125c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:24.896498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1aaf-7016-49b8-85d3-8431874125c1 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:24.897144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a1aaf-7016-49b8-85d3-8431874125c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:24.898564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a1aaf-7016-49b8-85d3-8431874125c1 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:25.012575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 450/890] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:09:24 2022] POST /identity/v3/users => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:09:27.147769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa74c-c863-44af-90ea-2efe0394f2f3 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:27.148295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa74c-c863-44af-90ea-2efe0394f2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:27.148405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fa74c-c863-44af-90ea-2efe0394f2f3 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:27.149670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 441/891] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:27.154763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4627a-acac-43ed-b4cb-95a1f401ff65 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:27.155197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4627a-acac-43ed-b4cb-95a1f401ff65 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:27.155334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4627a-acac-43ed-b4cb-95a1f401ff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:27.162098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:27.173862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-11f4627a-acac-43ed-b4cb-95a1f401ff65 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:27.175034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-11f4627a-acac-43ed-b4cb-95a1f401ff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:27.239384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 451/892] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:27.504237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fa74c-c863-44af-90ea-2efe0394f2f3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:27.571423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:27.601671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fa74c-c863-44af-90ea-2efe0394f2f3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:27.632804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c529c1-6a14-477a-9d94-82660d1cdb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:27.633403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c529c1-6a14-477a-9d94-82660d1cdb2b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:27.633675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c529c1-6a14-477a-9d94-82660d1cdb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:27.633940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c529c1-6a14-477a-9d94-82660d1cdb2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:27.643823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c529c1-6a14-477a-9d94-82660d1cdb2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c9c529c1-6a14-477a-9d94-82660d1cdb2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:09:27.648980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:09:27.650155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:09:27.651402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 442/893] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:09:27.655869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4627a-acac-43ed-b4cb-95a1f401ff65 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:27.749059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4627a-acac-43ed-b4cb-95a1f401ff65 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:27.777457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:27.781861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:27.782780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:27.782780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:27.782952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:27.783584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:09:27.783881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:27.785243 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:27.793383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 452/894] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:27.799140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c529c1-6a14-477a-9d94-82660d1cdb2b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:27.893810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c529c1-6a14-477a-9d94-82660d1cdb2b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:27.921749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:27.926365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c04c8-ac8f-464a-b12a-1c1753331b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:27.926987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c04c8-ac8f-464a-b12a-1c1753331b7b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:27.927262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c04c8-ac8f-464a-b12a-1c1753331b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:27.927531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c04c8-ac8f-464a-b12a-1c1753331b7b None admin] PATH_INFO: `/v3/users/5093c414d0ed40ccae94951cfa30375f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:27.939895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c04c8-ac8f-464a-b12a-1c1753331b7b None admin] RBAC: Authorizing `identity:get_user(user_id=5093c414d0ed40ccae94951cfa30375f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:27.941342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c04c8-ac8f-464a-b12a-1c1753331b7b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:27.943511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 443/895] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:09:27 2022] GET /identity/v3/users/5093c414d0ed40ccae94951cfa30375f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:27.949119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:28.045856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8348c-1219-41e2-b5f8-9daf94a53775 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:28.077560 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad5018-e5ba-4721-80bd-7f776ae27661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:28.078164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad5018-e5ba-4721-80bd-7f776ae27661 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:28.078442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad5018-e5ba-4721-80bd-7f776ae27661 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:28.078712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad5018-e5ba-4721-80bd-7f776ae27661 None admin] PATH_INFO: `/v3/projects/d8626f2d651b45598ad35543455fe50a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:28.079799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad5018-e5ba-4721-80bd-7f776ae27661 None admin] RBAC: Authorizing `identity:get_project(project_id=d8626f2d651b45598ad35543455fe50a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:28.081252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad5018-e5ba-4721-80bd-7f776ae27661 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:28.082855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 453/896] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:09:27 2022] GET /identity/v3/projects/d8626f2d651b45598ad35543455fe50a => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:28.089318 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c04c8-ac8f-464a-b12a-1c1753331b7b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:28.185062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c04c8-ac8f-464a-b12a-1c1753331b7b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:28.219587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e28cc-2d3e-4b93-b6dd-d96c4796514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:28.220214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e28cc-2d3e-4b93-b6dd-d96c4796514d None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:28.220522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e28cc-2d3e-4b93-b6dd-d96c4796514d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:28.220799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e28cc-2d3e-4b93-b6dd-d96c4796514d None admin] PATH_INFO: `/v3/projects/d8626f2d651b45598ad35543455fe50a/users/5093c414d0ed40ccae94951cfa30375f/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:28.222652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e28cc-2d3e-4b93-b6dd-d96c4796514d None admin] RBAC: Authorizing `identity:create_grant(project_id=d8626f2d651b45598ad35543455fe50a, user_id=5093c414d0ed40ccae94951cfa30375f, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:28.224100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e28cc-2d3e-4b93-b6dd-d96c4796514d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:28.298328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 444/897] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:09:28 2022] PUT /identity/v3/projects/d8626f2d651b45598ad35543455fe50a/users/5093c414d0ed40ccae94951cfa30375f/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:09:30.386969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97378-0360-4832-b3c0-a0f8f507f689 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:30.387493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97378-0360-4832-b3c0-a0f8f507f689 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:30.387493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97378-0360-4832-b3c0-a0f8f507f689 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:30.388810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 454/898] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:09:30.394032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c0e8-b318-4842-a382-7f6b56331d81 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:30.394478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c0e8-b318-4842-a382-7f6b56331d81 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:30.394585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30c0e8-b318-4842-a382-7f6b56331d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:30.412643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fe30c0e8-b318-4842-a382-7f6b56331d81 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:09:30.418256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fe30c0e8-b318-4842-a382-7f6b56331d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:09:30.457445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:30.477737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 445/899] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:09:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:09:30.744555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d97378-0360-4832-b3c0-a0f8f507f689 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:30.804077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:30.842220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d97378-0360-4832-b3c0-a0f8f507f689 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:30.873706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676af44-f8cf-4de0-b577-3ac40a610c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:30.874301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676af44-f8cf-4de0-b577-3ac40a610c2e None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:30.874575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676af44-f8cf-4de0-b577-3ac40a610c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:30.874842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676af44-f8cf-4de0-b577-3ac40a610c2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:30.885380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676af44-f8cf-4de0-b577-3ac40a610c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-7676af44-f8cf-4de0-b577-3ac40a610c2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:09:30.889972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:09:30.891047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:09:30.892142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 455/900] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:09:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:09:30.896910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30c0e8-b318-4842-a382-7f6b56331d81 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:30.992479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30c0e8-b318-4842-a382-7f6b56331d81 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:31.024618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:31.025149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:31.025426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:31.025698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:31.026346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:09:31.026649 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:31.027989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:31.035670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 446/901] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:09:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:31.041849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676af44-f8cf-4de0-b577-3ac40a610c2e None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:31.138778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676af44-f8cf-4de0-b577-3ac40a610c2e None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:31.160244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:31.171338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8079d98-fe83-49b6-aa16-a84aff795696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:31.186188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8079d98-fe83-49b6-aa16-a84aff795696 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:31.186188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8079d98-fe83-49b6-aa16-a84aff795696 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:31.186188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8079d98-fe83-49b6-aa16-a84aff795696 None admin] PATH_INFO: `/v3/users/5093c414d0ed40ccae94951cfa30375f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:31.186188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8079d98-fe83-49b6-aa16-a84aff795696 None admin] RBAC: Authorizing `identity:get_user(user_id=5093c414d0ed40ccae94951cfa30375f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:31.186188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8079d98-fe83-49b6-aa16-a84aff795696 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:31.186188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 456/902] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Oct 24 13:09:31 2022] GET /identity/v3/users/5093c414d0ed40ccae94951cfa30375f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:31.186640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:31.225446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:31.279400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027443f-fd38-47e6-bf21-cc0c7d080009 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:31.311289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94cb60-0031-4ffe-8530-0a1ef43748a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:31.311873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94cb60-0031-4ffe-8530-0a1ef43748a8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:31.312151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94cb60-0031-4ffe-8530-0a1ef43748a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:31.312453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94cb60-0031-4ffe-8530-0a1ef43748a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:31.376018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94cb60-0031-4ffe-8530-0a1ef43748a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:31.377475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94cb60-0031-4ffe-8530-0a1ef43748a8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:31.416479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:31.471299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 447/903] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:09:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:09:31.477729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8079d98-fe83-49b6-aa16-a84aff795696 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:31.572476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8079d98-fe83-49b6-aa16-a84aff795696 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:31.605305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f41f6-195e-4553-9f8c-721fd2b497d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:31.605910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f41f6-195e-4553-9f8c-721fd2b497d4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:31.606187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f41f6-195e-4553-9f8c-721fd2b497d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:31.606456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f41f6-195e-4553-9f8c-721fd2b497d4 None admin] PATH_INFO: `/v3/projects/d8626f2d651b45598ad35543455fe50a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:31.607543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f41f6-195e-4553-9f8c-721fd2b497d4 None admin] RBAC: Authorizing `identity:get_project(project_id=d8626f2d651b45598ad35543455fe50a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:31.608983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f41f6-195e-4553-9f8c-721fd2b497d4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:31.610562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 457/904] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Oct 24 13:09:31 2022] GET /identity/v3/projects/d8626f2d651b45598ad35543455fe50a => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:31.616609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94cb60-0031-4ffe-8530-0a1ef43748a8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:09:31.711160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94cb60-0031-4ffe-8530-0a1ef43748a8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:09:31.743248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7e758-e37a-455b-a28a-94b4ae7f8d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:09:31.743829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7e758-e37a-455b-a28a-94b4ae7f8d53 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:09:31.744106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7e758-e37a-455b-a28a-94b4ae7f8d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:09:31.744402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7e758-e37a-455b-a28a-94b4ae7f8d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:09:31.745106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7e758-e37a-455b-a28a-94b4ae7f8d53 None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=d8626f2d651b45598ad35543455fe50a, user.id=5093c414d0ed40ccae94951cfa30375f) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:09:31.745410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7e758-e37a-455b-a28a-94b4ae7f8d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:09:31.746806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7e758-e37a-455b-a28a-94b4ae7f8d53 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:09:31.752444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 448/905] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:09:31 2022] GET /identity/v3/role_assignments?user.id=5093c414d0ed40ccae94951cfa30375f&scope.project.id=d8626f2d651b45598ad35543455fe50a&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:09:41.601298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=71 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:41.749891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:42.689129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e7143-29f0-46be-b399-d86ea6959971 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:42.689736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e7143-29f0-46be-b399-d86ea6959971 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:42.689736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e7143-29f0-46be-b399-d86ea6959971 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:42.691014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 458/906] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:42.696372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98676de-5eb7-4c88-b481-fdcc85fa3277 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:42.696853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98676de-5eb7-4c88-b481-fdcc85fa3277 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:42.696961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98676de-5eb7-4c88-b481-fdcc85fa3277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:42.715598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f98676de-5eb7-4c88-b481-fdcc85fa3277 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:42.716805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f98676de-5eb7-4c88-b481-fdcc85fa3277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:42.781451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 449/907] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:10:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:10:43.048008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e7143-29f0-46be-b399-d86ea6959971 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:43.147127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e7143-29f0-46be-b399-d86ea6959971 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:43.182029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:43.182642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:43.182921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:43.183191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:43.193403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:43.194825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:10:43.209453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:43.210612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:10:43.212185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:10:43.212185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:43.212185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:43.212185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 459/908] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:43.217587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98676de-5eb7-4c88-b481-fdcc85fa3277 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:43.312352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98676de-5eb7-4c88-b481-fdcc85fa3277 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:43.340761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:43.346448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf90eb2-792a-451c-bced-4f061fe4ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:43.354413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf90eb2-792a-451c-bced-4f061fe4ac3b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:43.354413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf90eb2-792a-451c-bced-4f061fe4ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:43.354413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf90eb2-792a-451c-bced-4f061fe4ac3b None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:43.354413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf90eb2-792a-451c-bced-4f061fe4ac3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:43.354413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf90eb2-792a-451c-bced-4f061fe4ac3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:43.354413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf90eb2-792a-451c-bced-4f061fe4ac3b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:43.360632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 450/909] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:43.366302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:43.417982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:43.463904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089f947-f315-45bc-a3ef-b1a8a05ffc28 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:43.495791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992571-6871-4371-a8e5-6e757bfdf909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:43.496403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992571-6871-4371-a8e5-6e757bfdf909 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:43.496680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992571-6871-4371-a8e5-6e757bfdf909 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:43.496948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992571-6871-4371-a8e5-6e757bfdf909 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:43.497578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51992571-6871-4371-a8e5-6e757bfdf909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:43.498993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51992571-6871-4371-a8e5-6e757bfdf909 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:43.580215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 460/910] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Oct 24 13:10:43 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:10:45.731680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d46bf2-3286-47a2-af30-9c1a89846c9a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:45.732233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d46bf2-3286-47a2-af30-9c1a89846c9a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:45.732233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d46bf2-3286-47a2-af30-9c1a89846c9a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:45.733562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 451/911] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:45.738775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592cd2-416c-4c25-b63d-e84dca867fb6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:45.742779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592cd2-416c-4c25-b63d-e84dca867fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:45.742779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592cd2-416c-4c25-b63d-e84dca867fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:45.757859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-46592cd2-416c-4c25-b63d-e84dca867fb6 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:45.759040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-46592cd2-416c-4c25-b63d-e84dca867fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:45.825222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 461/912] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:10:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:10:46.092839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d46bf2-3286-47a2-af30-9c1a89846c9a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:46.192843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d46bf2-3286-47a2-af30-9c1a89846c9a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:46.226995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd79a-90ac-46fa-b8a7-18398fc66795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:46.227616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd79a-90ac-46fa-b8a7-18398fc66795 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:46.227891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd79a-90ac-46fa-b8a7-18398fc66795 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:46.228165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd79a-90ac-46fa-b8a7-18398fc66795 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:46.238008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7bd79a-90ac-46fa-b8a7-18398fc66795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1b7bd79a-90ac-46fa-b8a7-18398fc66795 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:10:46.242743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:46.243846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:46.252488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 452/913] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:10:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:46.253412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46592cd2-416c-4c25-b63d-e84dca867fb6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:46.288551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:46.347039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46592cd2-416c-4c25-b63d-e84dca867fb6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:46.379847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:46.380466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:46.380748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:46.381020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:46.381670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] RBAC: Adding query filter params (name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:46.381973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:46.383818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:46.391573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 462/914] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:10:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:46.397332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd79a-90ac-46fa-b8a7-18398fc66795 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:46.449369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:46.498482 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd79a-90ac-46fa-b8a7-18398fc66795 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:46.531701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:46.532347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:46.532625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:46.532892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:46.543311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:46.544776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:10:46.559640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:46.560830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:10:46.561928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:10:46.561928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:46.561928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:46.561928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 453/915] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:46.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:46.660486 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:46.663285 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff457d-257d-404e-ba12-44cfdb183290 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:46.697333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:46.697937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:46.698214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:46.698477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:46.699114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:46.699418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:46.700858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:46.711960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 463/916] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:46.717939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:46.815872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16049ba-692b-4088-a6d1-ec2a32cbeb5f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:46.817347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:46.849443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:46.850056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:46.850333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:46.850604 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:46.857424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:46.858878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:10:46.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:46.872147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:46.873382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 454/917] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:46.877836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:46.976926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda6574-dfad-433b-a86b-1c1949b4529b None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:47.010208 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:47.010792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:47.011066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:47.011332 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:47.012034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:47.012354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:47.013757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:47.030164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 464/918] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Oct 24 13:10:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:47.036043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:47.135548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23c7ca-0370-40e5-b3aa-77ef420e1266 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:47.168406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:47.169018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:47.169293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:47.169564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:47.180334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:47.181792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:10:47.195179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:47.196299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:10:47.197451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:10:47.197451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:47.197451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:47.200414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 455/919] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:47.202216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:47.221043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:47.300626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af6914-3c80-4ca9-86ad-e76bf6b7f4e9 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:47.334141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:47.334733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:47.335008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:47.335275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:47.336753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:47.337052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:47.338660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:47.349521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 465/920] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:47.355900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:47.371835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:47.452880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f94f-7234-4fca-b293-a5cc2748e782 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:47.486800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:47.487414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:47.487692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:47.487975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:47.494401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:47.495824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:10:47.506600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:47.507638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:47.508762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 456/921] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:47.513753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:47.591399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:47.612518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c686242-02cb-4932-9d9b-c05d95f70283 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:47.643914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50d191-0787-4011-a8cc-6469d429776c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:47.644522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d191-0787-4011-a8cc-6469d429776c None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:47.644797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d191-0787-4011-a8cc-6469d429776c None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:47.645064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d191-0787-4011-a8cc-6469d429776c None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:47.645738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50d191-0787-4011-a8cc-6469d429776c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:47.646037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50d191-0787-4011-a8cc-6469d429776c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:47.647455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50d191-0787-4011-a8cc-6469d429776c None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:47.659591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 466/922] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:10:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:47.666252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:47.739582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:47.763457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c9f70-b1db-40ad-a922-1fcd0d44f98e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:47.795555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd8916-b164-4c6e-aa98-d19ff18acd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:47.796331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd8916-b164-4c6e-aa98-d19ff18acd17 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:47.796614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd8916-b164-4c6e-aa98-d19ff18acd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:47.796885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd8916-b164-4c6e-aa98-d19ff18acd17 None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/8dd54892919e41f9b1f2ad734dc554f3/roles/04a120a50aa648fea07c999a73a1c22c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:47.811306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd8916-b164-4c6e-aa98-d19ff18acd17 None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=8dd54892919e41f9b1f2ad734dc554f3, role_id=04a120a50aa648fea07c999a73a1c22c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:47.812749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd8916-b164-4c6e-aa98-d19ff18acd17 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:47.929800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 457/923] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:10:47 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/8dd54892919e41f9b1f2ad734dc554f3/roles/04a120a50aa648fea07c999a73a1c22c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:10:50.025722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d8244-dcca-42c1-af38-79532ddec3f1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:50.026297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d8244-dcca-42c1-af38-79532ddec3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:50.026297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d8244-dcca-42c1-af38-79532ddec3f1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:50.027582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 467/924] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:50.032969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a85f1-cc57-4c7f-8649-25e4d43f81f7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:50.033376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a85f1-cc57-4c7f-8649-25e4d43f81f7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:50.033519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a85f1-cc57-4c7f-8649-25e4d43f81f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:50.051459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-712a85f1-cc57-4c7f-8649-25e4d43f81f7 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:50.055329 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-712a85f1-cc57-4c7f-8649-25e4d43f81f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:50.118444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 458/925] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:10:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:10:50.391267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d8244-dcca-42c1-af38-79532ddec3f1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:50.488949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d8244-dcca-42c1-af38-79532ddec3f1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:50.525685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a1254-f913-4ff8-a2cb-108df2e36052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:50.526300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a1254-f913-4ff8-a2cb-108df2e36052 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:50.526579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a1254-f913-4ff8-a2cb-108df2e36052 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:50.526848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a1254-f913-4ff8-a2cb-108df2e36052 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:50.533482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:50.537185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a1254-f913-4ff8-a2cb-108df2e36052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-aa7a1254-f913-4ff8-a2cb-108df2e36052 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:10:50.541842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:50.542965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:50.544048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 468/926] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Oct 24 13:10:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:50.549314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a85f1-cc57-4c7f-8649-25e4d43f81f7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:50.603437 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:50.645980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a85f1-cc57-4c7f-8649-25e4d43f81f7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:50.678800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:50.679997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:50.680301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:50.680571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:50.681218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:50.681513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:50.682850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:50.692316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 459/927] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Oct 24 13:10:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:50.696750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a1254-f913-4ff8-a2cb-108df2e36052 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:50.794019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a1254-f913-4ff8-a2cb-108df2e36052 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:50.827370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:50.827972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:50.828246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:50.828540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:50.838484 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:50.839900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:10:50.855393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:50.861169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:10:50.862314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:10:50.862314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:50.862314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:50.862314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 469/928] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:50.862617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:50.959062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3341-cb6f-4c58-8c2f-8b9bf8a53fee None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:50.975372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:50.993516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:50.994103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:50.994383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:50.994674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:50.995296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:50.995595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:50.996972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:51.008044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 460/929] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:51.013751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:51.063870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:51.113677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532326ac-dcb9-49ce-a795-01eb4097d5eb None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:51.145304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:51.145913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:51.146187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:51.146455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:51.154962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:51.156425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:10:51.167918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:51.169095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:51.170134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 470/930] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:51.174868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:51.270681 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1d994-82ed-42ca-a507-ab4c0f71744f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:51.304319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:51.304913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:51.305190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:51.305459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:51.306170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:51.306475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:51.307919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:51.324430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 461/931] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Oct 24 13:10:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:51.330381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:51.426150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970235b-8236-4ec8-9164-5dc27c83d403 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:51.435947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:51.458213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d033fc6-d978-463d-91c3-a8125badbfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:51.458791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033fc6-d978-463d-91c3-a8125badbfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:51.459066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033fc6-d978-463d-91c3-a8125badbfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:51.459328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033fc6-d978-463d-91c3-a8125badbfc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:51.523862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d033fc6-d978-463d-91c3-a8125badbfc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:51.525249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d033fc6-d978-463d-91c3-a8125badbfc6 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:51.623492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 471/932] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:10:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:10:51.629276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:51.665466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:51.730616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a8d83-bf30-4dcf-954d-3fda111b9027 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:51.763807 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:51.764475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:51.764755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:51.765024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:51.775585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:51.776989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:10:51.791438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:51.792565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:10:51.793675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:10:51.793675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:51.793675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:51.793675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 462/933] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:51.798319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d033fc6-d978-463d-91c3-a8125badbfc6 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:51.810159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:51.894430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d033fc6-d978-463d-91c3-a8125badbfc6 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:51.927328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:51.927920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:51.928310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:51.928501 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:51.929128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:51.929430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:51.931274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:51.942106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 472/934] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:51.948419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:52.030299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:52.044662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc918-8080-47eb-90dd-12cadcea529b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:52.078043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:52.078654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:52.078960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:52.079233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:52.086655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:52.088420 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:10:52.099733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:52.100788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:52.101969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 463/935] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:52.106692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:52.175154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:52.205026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214194c8-29b2-4910-bfdf-3e3eb9445909 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:52.237108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78a6cc-4ee9-41c0-a813-e66e0387fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:52.237704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78a6cc-4ee9-41c0-a813-e66e0387fde4 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:52.237981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78a6cc-4ee9-41c0-a813-e66e0387fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:52.238252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78a6cc-4ee9-41c0-a813-e66e0387fde4 None admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:52.238929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78a6cc-4ee9-41c0-a813-e66e0387fde4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:52.239234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78a6cc-4ee9-41c0-a813-e66e0387fde4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:52.240678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78a6cc-4ee9-41c0-a813-e66e0387fde4 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:52.252713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 473/936] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:52.258760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:52.359974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11f4aa-ca62-4667-a4bf-7df3bd7cf46c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:52.392034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ea9d9-6a69-43d1-902b-2966581d45ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:52.392647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ea9d9-6a69-43d1-902b-2966581d45ba None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:52.392920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ea9d9-6a69-43d1-902b-2966581d45ba None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:52.393189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ea9d9-6a69-43d1-902b-2966581d45ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:52.393895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ea9d9-6a69-43d1-902b-2966581d45ba None admin] RBAC: Adding query filter params (role.id=04a120a50aa648fea07c999a73a1c22c, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=8dd54892919e41f9b1f2ad734dc554f3) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:52.394198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ea9d9-6a69-43d1-902b-2966581d45ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:52.395605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ea9d9-6a69-43d1-902b-2966581d45ba None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:52.401181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 464/937] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:10:52 2022] GET /identity/v3/role_assignments?user.id=8dd54892919e41f9b1f2ad734dc554f3&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=04a120a50aa648fea07c999a73a1c22c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:54.550755 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67c9dc-c1f9-453f-b8d7-0b1f4de8a4c6 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:54.551347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67c9dc-c1f9-453f-b8d7-0b1f4de8a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:54.551347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67c9dc-c1f9-453f-b8d7-0b1f4de8a4c6 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:54.552702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 474/938] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:54.557956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22750-3102-4fb4-a994-0fa19163fe1c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:54.558377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22750-3102-4fb4-a994-0fa19163fe1c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:54.558501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22750-3102-4fb4-a994-0fa19163fe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:54.577164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-46e22750-3102-4fb4-a994-0fa19163fe1c None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:54.578334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-46e22750-3102-4fb4-a994-0fa19163fe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:54.581074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:54.643758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 465/939] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:10:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:10:54.907206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67c9dc-c1f9-453f-b8d7-0b1f4de8a4c6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:55.006268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67c9dc-c1f9-453f-b8d7-0b1f4de8a4c6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:55.034583 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:55.040131 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47bc84-5207-4254-a698-e64b9dd3d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:55.040772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47bc84-5207-4254-a698-e64b9dd3d295 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:55.041077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47bc84-5207-4254-a698-e64b9dd3d295 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:55.041346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47bc84-5207-4254-a698-e64b9dd3d295 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:55.051149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47bc84-5207-4254-a698-e64b9dd3d295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-5c47bc84-5207-4254-a698-e64b9dd3d295 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:10:55.056523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:55.060350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:55.061446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 475/940] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:10:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:55.063760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22750-3102-4fb4-a994-0fa19163fe1c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:55.160105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22750-3102-4fb4-a994-0fa19163fe1c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:55.194108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:55.194686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:55.194963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:55.195731 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:55.196439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:55.196727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:55.198067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:55.205768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 466/941] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:10:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:55.212740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47bc84-5207-4254-a698-e64b9dd3d295 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:55.309824 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47bc84-5207-4254-a698-e64b9dd3d295 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:55.343061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:55.343665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:55.343940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:55.344254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:55.354114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:55.355568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:10:55.369724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:55.370773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:10:55.371679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:10:55.371679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:55.371679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:55.371679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 476/942] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:55.376612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:55.379418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:55.473649 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1ccc-6965-4694-8a5e-62acc93024cb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:55.506501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:55.507097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:55.507388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:55.508164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:55.508834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:55.509136 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:55.510484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:55.522210 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 467/943] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:55.528483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:55.571663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:55.626045 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a218e-0868-43c6-a29e-fd24689775ac None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:55.658635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:55.659245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:55.659521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:55.659785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:55.667095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:55.668536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:10:55.679957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:55.681090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:55.682446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 477/944] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:55.686714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:55.747137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:55.783484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be7ca8-2c5c-4702-91c9-e04bf5c5f73b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:55.816101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:55.816723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:55.817062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:55.817334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:55.818050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:55.818354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:55.819813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:55.836849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 468/945] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Oct 24 13:10:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:55.842850 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:55.941500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3b9c8-6e00-4b4c-a373-02e41082cc10 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:55.942989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:55.974136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:55.974770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:55.975062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:55.975337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:55.986153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:55.987582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:10:56.001800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:56.002870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:10:56.003765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:10:56.003765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:56.003765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:56.003765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 478/946] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:56.009220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:56.105542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593cf15-621b-4b60-a14b-3a39819b597c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:56.126108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:56.139716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:56.140457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:56.140615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:56.140882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:56.141509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:56.141807 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:56.143145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:56.153945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 469/947] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:56.159953 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:56.260480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1e5d3-92dc-4692-81d4-26f441856651 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:56.294205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:56.297519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:56.297519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:56.297519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:56.301959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:56.303383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:10:56.314458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:56.315485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:10:56.316537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 479/948] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:10:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:56.321576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:56.419886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec3311-34f6-4ad1-91e0-4bc78f313453 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:56.452287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c8871-1bde-4938-ac5c-0adbe3f6abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:56.452905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8871-1bde-4938-ac5c-0adbe3f6abe9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:56.453184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8871-1bde-4938-ac5c-0adbe3f6abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:56.453462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c8871-1bde-4938-ac5c-0adbe3f6abe9 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:56.454137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c8871-1bde-4938-ac5c-0adbe3f6abe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:56.454439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c8871-1bde-4938-ac5c-0adbe3f6abe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:56.455903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c8871-1bde-4938-ac5c-0adbe3f6abe9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:56.468192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 470/949] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:10:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:56.474198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:56.490801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:56.574414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e28c0-edce-4006-9fe4-45d748d02c90 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:56.609412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f095256-692e-431a-b0cb-c11cbea1d353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:56.610057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f095256-692e-431a-b0cb-c11cbea1d353 None admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:56.610333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f095256-692e-431a-b0cb-c11cbea1d353 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:56.610603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f095256-692e-431a-b0cb-c11cbea1d353 None admin] PATH_INFO: `/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/8dd54892919e41f9b1f2ad734dc554f3/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:56.612489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f095256-692e-431a-b0cb-c11cbea1d353 None admin] RBAC: Authorizing `identity:create_grant(project_id=7508e18ad6cf4b14afba4f86574a19e0, user_id=8dd54892919e41f9b1f2ad734dc554f3, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:56.613906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f095256-692e-431a-b0cb-c11cbea1d353 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:56.718550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 480/950] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Oct 24 13:10:56 2022] PUT /identity/v3/projects/7508e18ad6cf4b14afba4f86574a19e0/users/8dd54892919e41f9b1f2ad734dc554f3/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:10:58.812822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7e31-cbc6-4469-9412-78f7d2b2f952 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:58.813419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7e31-cbc6-4469-9412-78f7d2b2f952 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:58.813419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c7e31-cbc6-4469-9412-78f7d2b2f952 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:58.814709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 471/951] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:58.820482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7b080-72b7-4d4e-90ea-fc7afe6f3af4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:58.820903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7b080-72b7-4d4e-90ea-fc7afe6f3af4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:58.821027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7b080-72b7-4d4e-90ea-fc7afe6f3af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:58.838908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bfc7b080-72b7-4d4e-90ea-fc7afe6f3af4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:58.840063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bfc7b080-72b7-4d4e-90ea-fc7afe6f3af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:58.903412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 481/952] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:10:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:10:59.168154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7e31-cbc6-4469-9412-78f7d2b2f952 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:59.210766 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:59.267379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c7e31-cbc6-4469-9412-78f7d2b2f952 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:59.300802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bea8eb-ff80-47b9-9548-6f3833d9e735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:59.301417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bea8eb-ff80-47b9-9548-6f3833d9e735 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:59.301695 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bea8eb-ff80-47b9-9548-6f3833d9e735 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:59.301994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bea8eb-ff80-47b9-9548-6f3833d9e735 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:59.311856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bea8eb-ff80-47b9-9548-6f3833d9e735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-78bea8eb-ff80-47b9-9548-6f3833d9e735 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:10:59.316613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:59.317742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:59.321487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 472/953] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Oct 24 13:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:59.323885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7b080-72b7-4d4e-90ea-fc7afe6f3af4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:59.326960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:59.422322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7b080-72b7-4d4e-90ea-fc7afe6f3af4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:59.454975 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:59.455565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:59.460905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:59.460905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:59.460905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:59.460905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:59.460905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:59.467063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 482/954] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Oct 24 13:10:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:59.472973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bea8eb-ff80-47b9-9548-6f3833d9e735 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:59.569174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:59.572091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bea8eb-ff80-47b9-9548-6f3833d9e735 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:59.605678 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:59.606290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:59.606570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:59.606841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:59.616428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:59.617888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:10:59.631893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:59.633602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:10:59.634830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:10:59.634830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:10:59.634830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:59.634830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 473/955] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:59.638967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:59.704779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:59.739027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788c978-287c-44ac-a32b-4fcafd6deb86 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:59.772321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:59.772929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:59.773207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:59.773475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:59.774136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:10:59.774438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:59.775778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:59.787013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 483/956] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:10:59.793249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:59.890561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57afff2-18f9-4272-bd08-d744317d0b52 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:59.923481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:59.924093 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:59.924400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:59.924675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:59.932528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:59.933946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:10:59.944708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:10:59.946002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:10:59.947213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 474/957] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:10:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:10:59.951557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:00.052185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b1a2d-19ab-4d5f-8f43-ed324e025dbc None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:00.076845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:00.085860 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:00.086474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:00.086750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:00.087278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:00.087996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:00.088325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:00.089746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:00.107380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 484/958] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Oct 24 13:10:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:11:00.113193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:00.128169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:00.210174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0902c-7e13-4c4d-89a0-6704f7ed603f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:00.244646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a091c-e1be-4166-ae56-44c075d6a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:00.245236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a091c-e1be-4166-ae56-44c075d6a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:00.245512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a091c-e1be-4166-ae56-44c075d6a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:00.245782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a091c-e1be-4166-ae56-44c075d6a2e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:00.312728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a091c-e1be-4166-ae56-44c075d6a2e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:00.314238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a091c-e1be-4166-ae56-44c075d6a2e2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:00.327804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:00.408963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 475/959] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Oct 24 13:11:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:11:00.415346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:00.511021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fcd65-a918-43f4-be52-7b1c9e5ba40f None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:00.543086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:00.543695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:00.543972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:00.544241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:00.553715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:00.555201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:11:00.569437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:11:00.570984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:11:00.572006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:11:00.572006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:11:00.572006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:11:00.572006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 485/960] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:11:00.576747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a091c-e1be-4166-ae56-44c075d6a2e2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:00.673683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a091c-e1be-4166-ae56-44c075d6a2e2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:00.685848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:00.706414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:00.707004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:00.707282 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:00.707583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:00.708234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:00.708565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:00.709929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:00.722112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 476/961] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Oct 24 13:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:11:00.728154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:00.764262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:00.827843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b90653-f172-4b40-bfeb-b9ac6ed21691 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:00.862822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:00.863441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:00.863720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:00.864011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:00.871028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:00.872479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:11:00.885037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application with Lock( Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:11:00.885946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: ERROR keystone.server.flask.application Oct 24 13:11:00.886896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 486/962] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Oct 24 13:11:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:11:00.891993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:00.989513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5580f-9f22-41fa-a7ac-10084248b34a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:01.023080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8b865-4e66-46b4-aebc-a8f1fd3d6446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:01.023673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8b865-4e66-46b4-aebc-a8f1fd3d6446 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:01.023955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8b865-4e66-46b4-aebc-a8f1fd3d6446 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:01.024230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8b865-4e66-46b4-aebc-a8f1fd3d6446 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:01.024951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8b865-4e66-46b4-aebc-a8f1fd3d6446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:01.030275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8b865-4e66-46b4-aebc-a8f1fd3d6446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:01.030275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8b865-4e66-46b4-aebc-a8f1fd3d6446 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:01.037705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 477/963] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Oct 24 13:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:11:01.043796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:01.138292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:01.143916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623acfb9-9bd1-4d60-bc7f-845a66dd1893 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:01.176283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ec86a-150f-40fe-8860-9c2d7ce7ca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:01.176880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ec86a-150f-40fe-8860-9c2d7ce7ca88 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:01.177155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ec86a-150f-40fe-8860-9c2d7ce7ca88 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:01.177421 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ec86a-150f-40fe-8860-9c2d7ce7ca88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:01.178129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ec86a-150f-40fe-8860-9c2d7ce7ca88 None admin] RBAC: Adding query filter params (role.id=15df2a7bb32b484894972c74c547f5a4, scope.project.id=7508e18ad6cf4b14afba4f86574a19e0, user.id=8dd54892919e41f9b1f2ad734dc554f3) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:01.178430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ec86a-150f-40fe-8860-9c2d7ce7ca88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:01.180441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ec86a-150f-40fe-8860-9c2d7ce7ca88 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:01.185957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 487/964] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Oct 24 13:11:01 2022] GET /identity/v3/role_assignments?user.id=8dd54892919e41f9b1f2ad734dc554f3&scope.project.id=7508e18ad6cf4b14afba4f86574a19e0&role.id=15df2a7bb32b484894972c74c547f5a4 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:11:03.334228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77f31d-5475-4d04-bd2b-ca8e5b173c29 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:03.334785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77f31d-5475-4d04-bd2b-ca8e5b173c29 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:03.334785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77f31d-5475-4d04-bd2b-ca8e5b173c29 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:03.336647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 478/965] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:11:03.341907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b1241-b8af-404a-b09a-ee2a513ae768 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:03.342356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b1241-b8af-404a-b09a-ee2a513ae768 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:03.342455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b1241-b8af-404a-b09a-ee2a513ae768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:03.360758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-cb6b1241-b8af-404a-b09a-ee2a513ae768 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:11:03.363445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cb6b1241-b8af-404a-b09a-ee2a513ae768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:11:03.429132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 488/966] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:11:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:11:03.698003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77f31d-5475-4d04-bd2b-ca8e5b173c29 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:03.729405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:03.796714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77f31d-5475-4d04-bd2b-ca8e5b173c29 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:03.836415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:03.837291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:03.837572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:03.837844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:03.838417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:03.839771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:11:03.848369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:11:03.850674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:03.851648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 479/967] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Oct 24 13:11:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:11:03.855459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b1241-b8af-404a-b09a-ee2a513ae768 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:03.941188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:03.950902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b1241-b8af-404a-b09a-ee2a513ae768 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:03.982873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ece6c-ba05-41c1-9048-94bb0cc7d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:03.983456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ece6c-ba05-41c1-9048-94bb0cc7d315 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:03.983733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ece6c-ba05-41c1-9048-94bb0cc7d315 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:03.984000 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ece6c-ba05-41c1-9048-94bb0cc7d315 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:03.984637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ece6c-ba05-41c1-9048-94bb0cc7d315 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:03.984938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ece6c-ba05-41c1-9048-94bb0cc7d315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:03.986289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ece6c-ba05-41c1-9048-94bb0cc7d315 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:03.990926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 489/968] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Oct 24 13:11:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:11:03.997316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:04.090667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:04.096491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e90597-8543-42ab-9d21-2456e4add233 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:04.129444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d735ec-2ff5-4a29-a7c4-5b77083be4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:04.130031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d735ec-2ff5-4a29-a7c4-5b77083be4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:04.130306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d735ec-2ff5-4a29-a7c4-5b77083be4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:04.130573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d735ec-2ff5-4a29-a7c4-5b77083be4d5 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:04.131189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d735ec-2ff5-4a29-a7c4-5b77083be4d5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:04.131491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d735ec-2ff5-4a29-a7c4-5b77083be4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:04.136307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d735ec-2ff5-4a29-a7c4-5b77083be4d5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:04.137960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 480/969] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Oct 24 13:11:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:11:06.296943 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7d6e-bfee-4256-82db-cffbbef55225 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:06.297468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7d6e-bfee-4256-82db-cffbbef55225 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:06.297468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7d6e-bfee-4256-82db-cffbbef55225 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:06.298876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 490/970] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:11:06.304734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e3d5-5482-40d7-90ee-932dbe3bb906 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:06.305175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e3d5-5482-40d7-90ee-932dbe3bb906 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:06.305366 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e3d5-5482-40d7-90ee-932dbe3bb906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:06.323908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1d72e3d5-5482-40d7-90ee-932dbe3bb906 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:11:06.326374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1d72e3d5-5482-40d7-90ee-932dbe3bb906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:11:06.390260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 481/971] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:11:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:11:06.659716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a7d6e-bfee-4256-82db-cffbbef55225 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:06.760038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a7d6e-bfee-4256-82db-cffbbef55225 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:06.793554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5ad19-b1ff-48f8-85af-3b79f3a60cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:06.794138 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5ad19-b1ff-48f8-85af-3b79f3a60cf2 None admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:06.794416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5ad19-b1ff-48f8-85af-3b79f3a60cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:06.794689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5ad19-b1ff-48f8-85af-3b79f3a60cf2 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:06.795354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5ad19-b1ff-48f8-85af-3b79f3a60cf2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:06.797472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5ad19-b1ff-48f8-85af-3b79f3a60cf2 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:06.912474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 491/972] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Oct 24 13:11:06 2022] POST /identity/v3/services => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:11:09.049683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdf3a2-24a6-4d90-8888-fd3eb3a096f1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:09.050241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdf3a2-24a6-4d90-8888-fd3eb3a096f1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:09.050241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdf3a2-24a6-4d90-8888-fd3eb3a096f1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:09.051537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 482/973] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:11:09.057234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef119-6e4c-4ec5-890f-994eeca20cec None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:09.057646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef119-6e4c-4ec5-890f-994eeca20cec None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:09.057828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef119-6e4c-4ec5-890f-994eeca20cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:09.076393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7d7ef119-6e4c-4ec5-890f-994eeca20cec None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:11:09.077574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7d7ef119-6e4c-4ec5-890f-994eeca20cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:11:09.087047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:09.147566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 492/974] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:11:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:11:09.418515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdf3a2-24a6-4d90-8888-fd3eb3a096f1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:09.508434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:09.518485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdf3a2-24a6-4d90-8888-fd3eb3a096f1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:09.552383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:09.553011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:09.553265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:09.553530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:09.554705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:09.556040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:11:09.564635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:11:09.565764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:09.566632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 483/975] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Oct 24 13:11:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:11:09.573461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ef119-6e4c-4ec5-890f-994eeca20cec None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:09.668382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ef119-6e4c-4ec5-890f-994eeca20cec None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:09.702226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:09.702748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:09.703021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:09.703287 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:09.703899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:09.704199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:09.705578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:09.711121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 493/976] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Oct 24 13:11:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:11:09.717434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:09.815136 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b7096-58fc-451b-a200-796d2051d49e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:09.848251 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223b67b-7b68-419e-ba24-0e84ab051abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:09.848870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b67b-7b68-419e-ba24-0e84ab051abc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:09.849148 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b67b-7b68-419e-ba24-0e84ab051abc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:09.849418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b67b-7b68-419e-ba24-0e84ab051abc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:09.850108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223b67b-7b68-419e-ba24-0e84ab051abc None admin] RBAC: Adding query filter params (interface=public, service_id=4d7abb22e8a34c3194220c8293e1eb22, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:09.850415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223b67b-7b68-419e-ba24-0e84ab051abc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:09.851757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223b67b-7b68-419e-ba24-0e84ab051abc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:09.856648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 484/977] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Oct 24 13:11:09 2022] GET /identity/v3/endpoints?service_id=4d7abb22e8a34c3194220c8293e1eb22&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:11:09.862523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:09.866872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:09.959026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935ce0d-9a7d-4d54-a7ba-afaeadf48664 None admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:09.992038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b461-f001-444d-9ff9-3cb6b4838032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:09.992650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b461-f001-444d-9ff9-3cb6b4838032 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:09.992929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b461-f001-444d-9ff9-3cb6b4838032 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:09.993193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b461-f001-444d-9ff9-3cb6b4838032 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:09.993775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2b461-f001-444d-9ff9-3cb6b4838032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:09.995120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2b461-f001-444d-9ff9-3cb6b4838032 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:10.001735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 494/978] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Oct 24 13:11:09 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:11:12.137186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5fbef-ca64-4dd7-b4b1-50931c3c5c07 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:12.137672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5fbef-ca64-4dd7-b4b1-50931c3c5c07 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:12.137765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5fbef-ca64-4dd7-b4b1-50931c3c5c07 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:12.139049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 485/979] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:11:12.144337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bb290-cf95-4614-8488-89d847811351 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:12.144761 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bb290-cf95-4614-8488-89d847811351 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:12.144871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bb290-cf95-4614-8488-89d847811351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:12.163125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f35bb290-cf95-4614-8488-89d847811351 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:11:12.164335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f35bb290-cf95-4614-8488-89d847811351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:11:12.224153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:12.229541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 495/980] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:11:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:11:12.495976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5fbef-ca64-4dd7-b4b1-50931c3c5c07 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:12.515303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:12.595752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5fbef-ca64-4dd7-b4b1-50931c3c5c07 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:12.632146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:12.632792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:12.633069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:12.633338 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:12.633965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:12.635312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:11:12.644180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:11:12.645433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Oct 24 13:11:12.646309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 486/981] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Oct 24 13:11:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:11:12.651404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bb290-cf95-4614-8488-89d847811351 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:12.747425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bb290-cf95-4614-8488-89d847811351 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:12.780177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711345a-d659-4bf2-a9b9-79ab1b912d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:12.780795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711345a-d659-4bf2-a9b9-79ab1b912d68 None admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:12.781123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711345a-d659-4bf2-a9b9-79ab1b912d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:12.781472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711345a-d659-4bf2-a9b9-79ab1b912d68 None admin] PATH_INFO: `/v3/services` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:12.782082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711345a-d659-4bf2-a9b9-79ab1b912d68 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:11:12.782379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711345a-d659-4bf2-a9b9-79ab1b912d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:12.783714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711345a-d659-4bf2-a9b9-79ab1b912d68 None admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:12.789342 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 496/982] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Oct 24 13:11:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:11:12.795065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:11:12.878745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:12.895730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c4852-1595-4d6d-8afb-c7a46768c3a8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:11:12.929264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b137e-54a4-4fd2-a279-2fae6615fa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:11:12.929879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b137e-54a4-4fd2-a279-2fae6615fa58 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:11:12.930142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b137e-54a4-4fd2-a279-2fae6615fa58 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:11:12.930413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b137e-54a4-4fd2-a279-2fae6615fa58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:11:12.931063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b137e-54a4-4fd2-a279-2fae6615fa58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:11:12.932436 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b137e-54a4-4fd2-a279-2fae6615fa58 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:11:13.026304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 487/983] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:11:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:11:22.785863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=69 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:11:23.021295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:04.376679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4764b0-ffe6-4c09-af81-91c040ec63cb None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:04.377121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4764b0-ffe6-4c09-af81-91c040ec63cb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:04.377214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4764b0-ffe6-4c09-af81-91c040ec63cb None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:04.378547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 497/984] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:17:04.384127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a9607-e8ea-437e-9025-1da7625b3ffb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:04.384642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a9607-e8ea-437e-9025-1da7625b3ffb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:04.384723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a9607-e8ea-437e-9025-1da7625b3ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:04.471321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6e8a9607-e8ea-437e-9025-1da7625b3ffb None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:04.472593 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6e8a9607-e8ea-437e-9025-1da7625b3ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:04.604867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 488/985] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:04.899264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de83f7-d668-49aa-8ad8-edfdfe0c696a None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:04.899952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de83f7-d668-49aa-8ad8-edfdfe0c696a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:04.900515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de83f7-d668-49aa-8ad8-edfdfe0c696a None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:04.902456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 498/986] 10.222.0.48 () {54 vars in 964 bytes} [Mon Oct 24 13:17:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:04.908114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e3c2-b751-4d7c-87f0-0d7a911e4189 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:04.908448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e3c2-b751-4d7c-87f0-0d7a911e4189 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:04.908735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e3c2-b751-4d7c-87f0-0d7a911e4189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:04.998719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6d55e3c2-b751-4d7c-87f0-0d7a911e4189 None None] MFA Rules not processed for user `65d4ca1062fe496dbe16d6cd1bcac1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:04.999990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6d55e3c2-b751-4d7c-87f0-0d7a911e4189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:05.023976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:05.105406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de83f7-d668-49aa-8ad8-edfdfe0c696a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:05.107286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-83de83f7-d668-49aa-8ad8-edfdfe0c696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:05.112319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 489/987] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Oct 24 13:17:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:05.229148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de83f7-d668-49aa-8ad8-edfdfe0c696a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:05.322185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ca399-40c7-4ee2-bf17-f6fbbd95957a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:05.323125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ca399-40c7-4ee2-bf17-f6fbbd95957a service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:05.323556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ca399-40c7-4ee2-bf17-f6fbbd95957a service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:05.324257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ca399-40c7-4ee2-bf17-f6fbbd95957a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:05.326580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cb8ca399-40c7-4ee2-bf17-f6fbbd95957a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:05.402025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:05.471320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ca399-40c7-4ee2-bf17-f6fbbd95957a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:05.473138 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ca399-40c7-4ee2-bf17-f6fbbd95957a service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:05.493424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 499/988] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:08.029171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b201-6ba0-4e18-a396-00f200fe115e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:08.029739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b201-6ba0-4e18-a396-00f200fe115e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:08.029739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961b201-6ba0-4e18-a396-00f200fe115e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:08.031038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 490/989] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:08.036188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0de6a-46b6-44be-8e62-02ee96cbf034 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:08.036655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0de6a-46b6-44be-8e62-02ee96cbf034 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:08.036925 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0de6a-46b6-44be-8e62-02ee96cbf034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:08.112805 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d7b0de6a-46b6-44be-8e62-02ee96cbf034 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:08.114047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d7b0de6a-46b6-44be-8e62-02ee96cbf034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:08.216512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 500/990] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:08.492256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961b201-6ba0-4e18-a396-00f200fe115e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:08.493807 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e961b201-6ba0-4e18-a396-00f200fe115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:08.599055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961b201-6ba0-4e18-a396-00f200fe115e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:08.673878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:08.676206 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e6eaf-756c-4fcf-8d3c-9b682901a0e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:08.676762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e6eaf-756c-4fcf-8d3c-9b682901a0e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:08.677036 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e6eaf-756c-4fcf-8d3c-9b682901a0e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:08.677303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e6eaf-756c-4fcf-8d3c-9b682901a0e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:08.678908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-188e6eaf-756c-4fcf-8d3c-9b682901a0e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:08.789684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e6eaf-756c-4fcf-8d3c-9b682901a0e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:08.791047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e6eaf-756c-4fcf-8d3c-9b682901a0e1 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:08.809087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 491/991] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:11.387825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593b23-1bca-43a7-884b-f281a249abe6 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:11.388358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593b23-1bca-43a7-884b-f281a249abe6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:11.388358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593b23-1bca-43a7-884b-f281a249abe6 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:11.389637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 501/992] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:11.394873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b2cb1-2c2a-4018-afc8-d6323a76c432 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:11.395320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b2cb1-2c2a-4018-afc8-d6323a76c432 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:11.395419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b2cb1-2c2a-4018-afc8-d6323a76c432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:11.463258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-233b2cb1-2c2a-4018-afc8-d6323a76c432 None None] MFA Rules not processed for user `94f198dd02e44352b78185bbf6b74125`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:11.464735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-233b2cb1-2c2a-4018-afc8-d6323a76c432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:11.515827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:11.559062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 492/993] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:11.842754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88593b23-1bca-43a7-884b-f281a249abe6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:11.844291 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-88593b23-1bca-43a7-884b-f281a249abe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:11.854423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:11.945584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88593b23-1bca-43a7-884b-f281a249abe6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:12.016449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68ba1-887b-4fbe-9b03-4e6690516cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:12.017058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68ba1-887b-4fbe-9b03-4e6690516cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:12.017334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68ba1-887b-4fbe-9b03-4e6690516cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:12.017604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68ba1-887b-4fbe-9b03-4e6690516cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:12.019199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-afc68ba1-887b-4fbe-9b03-4e6690516cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:12.081313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:12.127648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc68ba1-887b-4fbe-9b03-4e6690516cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:12.129101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc68ba1-887b-4fbe-9b03-4e6690516cf7 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:12.146847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 502/994] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:15.042439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1674-efef-4663-af34-9cfc10a35a08 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:15.042976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1674-efef-4663-af34-9cfc10a35a08 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:15.042976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1674-efef-4663-af34-9cfc10a35a08 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:15.044323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 493/995] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:15.049396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79082098-89c4-4983-a77a-77a34b4a0236 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:15.049784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79082098-89c4-4983-a77a-77a34b4a0236 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:15.049938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79082098-89c4-4983-a77a-77a34b4a0236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:15.114445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-79082098-89c4-4983-a77a-77a34b4a0236 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:15.115643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-79082098-89c4-4983-a77a-77a34b4a0236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:15.208892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 503/996] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:15.511479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008f5d4-f72e-4499-a697-77fb85f34358 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:15.511764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008f5d4-f72e-4499-a697-77fb85f34358 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:15.512035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008f5d4-f72e-4499-a697-77fb85f34358 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:15.513403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 494/997] 10.222.0.48 () {54 vars in 964 bytes} [Mon Oct 24 13:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:15.519218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6e8b-fb6d-457e-8e13-9b0a6c7391d3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:15.519508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6e8b-fb6d-457e-8e13-9b0a6c7391d3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:15.519786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6e8b-fb6d-457e-8e13-9b0a6c7391d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:15.542011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:15.595750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-341b6e8b-fb6d-457e-8e13-9b0a6c7391d3 None None] MFA Rules not processed for user `65d4ca1062fe496dbe16d6cd1bcac1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:15.596977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-341b6e8b-fb6d-457e-8e13-9b0a6c7391d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:15.670101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 504/998] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Oct 24 13:17:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:15.679052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008f5d4-f72e-4499-a697-77fb85f34358 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:15.680594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3008f5d4-f72e-4499-a697-77fb85f34358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:15.786804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008f5d4-f72e-4499-a697-77fb85f34358 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:15.858261 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cbb7f-4c50-4d32-9df5-bda6ec54b27b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:15.858850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cbb7f-4c50-4d32-9df5-bda6ec54b27b service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:15.859127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cbb7f-4c50-4d32-9df5-bda6ec54b27b service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:15.859398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cbb7f-4c50-4d32-9df5-bda6ec54b27b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:15.860987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b09cbb7f-4c50-4d32-9df5-bda6ec54b27b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:15.868575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:15.967523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cbb7f-4c50-4d32-9df5-bda6ec54b27b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:15.968975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cbb7f-4c50-4d32-9df5-bda6ec54b27b service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:15.986483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 495/999] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:19.477418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076332d8-16a2-47da-8606-0faada99faa5 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:19.477959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076332d8-16a2-47da-8606-0faada99faa5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:19.477959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076332d8-16a2-47da-8606-0faada99faa5 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:19.479317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 505/1000] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:19.484756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df134ca0-a561-4d2c-8d3d-bb38b91f54d6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:19.485188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df134ca0-a561-4d2c-8d3d-bb38b91f54d6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:19.485291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df134ca0-a561-4d2c-8d3d-bb38b91f54d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:19.552808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-df134ca0-a561-4d2c-8d3d-bb38b91f54d6 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:19.554086 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-df134ca0-a561-4d2c-8d3d-bb38b91f54d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:19.617063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:19.659835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 496/1001] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:19.943126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076332d8-16a2-47da-8606-0faada99faa5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:19.945238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-076332d8-16a2-47da-8606-0faada99faa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:20.051408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076332d8-16a2-47da-8606-0faada99faa5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:20.064823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:20.126473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183a2cd-4aae-49c4-8bf4-bf8fd8eb5148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:20.127075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183a2cd-4aae-49c4-8bf4-bf8fd8eb5148 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:20.127378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183a2cd-4aae-49c4-8bf4-bf8fd8eb5148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:20.127650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183a2cd-4aae-49c4-8bf4-bf8fd8eb5148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:20.129266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5183a2cd-4aae-49c4-8bf4-bf8fd8eb5148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:20.236905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183a2cd-4aae-49c4-8bf4-bf8fd8eb5148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:20.238371 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183a2cd-4aae-49c4-8bf4-bf8fd8eb5148 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:20.255829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 506/1002] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:22.456178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff37bee-aadd-4bde-a5ae-4f05ed87b28b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:22.456774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff37bee-aadd-4bde-a5ae-4f05ed87b28b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:22.456774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff37bee-aadd-4bde-a5ae-4f05ed87b28b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:22.458133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 497/1003] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:22.464427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c9dba-b5ef-4b56-b974-175cee8f41ff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:22.464820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c9dba-b5ef-4b56-b974-175cee8f41ff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:22.465106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c9dba-b5ef-4b56-b974-175cee8f41ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:22.513385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:22.531656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-377c9dba-b5ef-4b56-b974-175cee8f41ff None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:22.532927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-377c9dba-b5ef-4b56-b974-175cee8f41ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:22.636504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 507/1004] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:22.919662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff37bee-aadd-4bde-a5ae-4f05ed87b28b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:22.921234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0ff37bee-aadd-4bde-a5ae-4f05ed87b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:23.024859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff37bee-aadd-4bde-a5ae-4f05ed87b28b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:23.101720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b8929-b718-4995-841f-395fd4b311aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:23.102341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b8929-b718-4995-841f-395fd4b311aa service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:23.102598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b8929-b718-4995-841f-395fd4b311aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:23.102865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b8929-b718-4995-841f-395fd4b311aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:23.104481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-672b8929-b718-4995-841f-395fd4b311aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:23.121238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:23.215401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b8929-b718-4995-841f-395fd4b311aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:23.216874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b8929-b718-4995-841f-395fd4b311aa service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:23.235703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 498/1005] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:17:25.433950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d564cb-4248-4961-a18e-b7d0db442e53 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:25.434460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d564cb-4248-4961-a18e-b7d0db442e53 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:25.434460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d564cb-4248-4961-a18e-b7d0db442e53 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:25.435795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 508/1006] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:25.441339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15b216-bd0e-4bf7-a0ef-6f9ba77e7f39 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:25.441772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15b216-bd0e-4bf7-a0ef-6f9ba77e7f39 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:25.441867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15b216-bd0e-4bf7-a0ef-6f9ba77e7f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:25.508305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4b15b216-bd0e-4bf7-a0ef-6f9ba77e7f39 None None] MFA Rules not processed for user `94f198dd02e44352b78185bbf6b74125`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:25.509486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4b15b216-bd0e-4bf7-a0ef-6f9ba77e7f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:25.574546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:25.606936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 499/1007] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:25.891823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d564cb-4248-4961-a18e-b7d0db442e53 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:25.893397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-26d564cb-4248-4961-a18e-b7d0db442e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:26.001015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d564cb-4248-4961-a18e-b7d0db442e53 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:26.023772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:26.076882 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9b4b6-e78b-447b-9cdd-08377ae4a835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:26.077392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b4b6-e78b-447b-9cdd-08377ae4a835 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:26.077664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b4b6-e78b-447b-9cdd-08377ae4a835 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:26.077934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b4b6-e78b-447b-9cdd-08377ae4a835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:26.079495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-edd9b4b6-e78b-447b-9cdd-08377ae4a835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:26.190330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9b4b6-e78b-447b-9cdd-08377ae4a835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:26.191779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9b4b6-e78b-447b-9cdd-08377ae4a835 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:26.209675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 509/1008] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:28.775252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338c07a-ab33-41cc-a9f8-19be58e740ed None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:28.775759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338c07a-ab33-41cc-a9f8-19be58e740ed None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:28.775759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338c07a-ab33-41cc-a9f8-19be58e740ed None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:28.777080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 500/1009] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:28.782343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf9479-8dc5-43df-8328-e94ede9599ab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:28.782758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf9479-8dc5-43df-8328-e94ede9599ab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:28.782872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf9479-8dc5-43df-8328-e94ede9599ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:28.844724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:28.847277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aecf9479-8dc5-43df-8328-e94ede9599ab None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:28.848503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aecf9479-8dc5-43df-8328-e94ede9599ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:28.943622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 510/1010] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:29.224307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338c07a-ab33-41cc-a9f8-19be58e740ed None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:29.225912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1338c07a-ab33-41cc-a9f8-19be58e740ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:29.326258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338c07a-ab33-41cc-a9f8-19be58e740ed None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:29.399551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f120d-ae16-4a60-bb00-8aa9d1b4faf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:29.400137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f120d-ae16-4a60-bb00-8aa9d1b4faf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:29.400440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f120d-ae16-4a60-bb00-8aa9d1b4faf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:29.400711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f120d-ae16-4a60-bb00-8aa9d1b4faf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:29.402253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a47f120d-ae16-4a60-bb00-8aa9d1b4faf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:29.428179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:29.509455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f120d-ae16-4a60-bb00-8aa9d1b4faf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:29.510881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f120d-ae16-4a60-bb00-8aa9d1b4faf3 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:29.528871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 501/1011] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:31.720099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecc8f-e713-4b87-84cf-ed22508c6bd2 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:31.720695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecc8f-e713-4b87-84cf-ed22508c6bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:31.720695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ecc8f-e713-4b87-84cf-ed22508c6bd2 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:31.722018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 511/1012] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:31.727975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f913d6-a0db-4474-810b-b62a3e06a9c0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:31.728405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f913d6-a0db-4474-810b-b62a3e06a9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:31.728568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f913d6-a0db-4474-810b-b62a3e06a9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:31.792949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-57f913d6-a0db-4474-810b-b62a3e06a9c0 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:31.794143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-57f913d6-a0db-4474-810b-b62a3e06a9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:31.866969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:31.888143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 502/1013] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:32.170379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ecc8f-e713-4b87-84cf-ed22508c6bd2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:32.171942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-386ecc8f-e713-4b87-84cf-ed22508c6bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:32.274667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ecc8f-e713-4b87-84cf-ed22508c6bd2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:32.302721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:32.348096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b364b74-bd98-40db-9a68-953adc54c2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:32.348650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b364b74-bd98-40db-9a68-953adc54c2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:32.348927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b364b74-bd98-40db-9a68-953adc54c2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:32.349195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b364b74-bd98-40db-9a68-953adc54c2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:32.351300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9b364b74-bd98-40db-9a68-953adc54c2b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:32.456551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b364b74-bd98-40db-9a68-953adc54c2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:32.457978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b364b74-bd98-40db-9a68-953adc54c2b3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:32.476676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 512/1014] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:35.186970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ebcb4-9291-44df-956b-a80dc633d7a3 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:35.187541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ebcb4-9291-44df-956b-a80dc633d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:35.187541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ebcb4-9291-44df-956b-a80dc633d7a3 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:35.188894 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 503/1015] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:35.194105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226feb6-b310-45f4-a45a-1e6703ffe084 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:35.195083 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226feb6-b310-45f4-a45a-1e6703ffe084 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:35.195180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226feb6-b310-45f4-a45a-1e6703ffe084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:35.259637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c226feb6-b310-45f4-a45a-1e6703ffe084 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:35.260851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c226feb6-b310-45f4-a45a-1e6703ffe084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:35.268910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:35.357788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 513/1016] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:35.649057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ebcb4-9291-44df-956b-a80dc633d7a3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:35.650612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-831ebcb4-9291-44df-956b-a80dc633d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:35.751587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ebcb4-9291-44df-956b-a80dc633d7a3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:35.824509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a044c2-c5a0-4e04-a1e5-1ab234d0905b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:35.825103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a044c2-c5a0-4e04-a1e5-1ab234d0905b service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:35.825382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a044c2-c5a0-4e04-a1e5-1ab234d0905b service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:35.825650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a044c2-c5a0-4e04-a1e5-1ab234d0905b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:35.827224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e8a044c2-c5a0-4e04-a1e5-1ab234d0905b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:35.862298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:35.933890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a044c2-c5a0-4e04-a1e5-1ab234d0905b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:35.935296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a044c2-c5a0-4e04-a1e5-1ab234d0905b service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:35.953362 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 504/1017] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:40.289737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fa898-f897-4257-8b68-ec970d6bb773 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:40.290328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fa898-f897-4257-8b68-ec970d6bb773 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:40.290328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fa898-f897-4257-8b68-ec970d6bb773 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:40.291617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 514/1018] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:40.296843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d95724-9319-4ac4-ad1e-567c12bba74b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:40.297287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d95724-9319-4ac4-ad1e-567c12bba74b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:40.297382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d95724-9319-4ac4-ad1e-567c12bba74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:40.363977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-44d95724-9319-4ac4-ad1e-567c12bba74b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:40.365218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-44d95724-9319-4ac4-ad1e-567c12bba74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:40.449848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:40.461384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 505/1019] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:40.742844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fa898-f897-4257-8b68-ec970d6bb773 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:40.744402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e90fa898-f897-4257-8b68-ec970d6bb773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:40.847363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fa898-f897-4257-8b68-ec970d6bb773 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:40.884895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:40.920918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76f27a-f5d4-4201-8dd5-a46ff90bbee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:40.921511 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76f27a-f5d4-4201-8dd5-a46ff90bbee1 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:40.921785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76f27a-f5d4-4201-8dd5-a46ff90bbee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:40.922053 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76f27a-f5d4-4201-8dd5-a46ff90bbee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:40.923623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ef76f27a-f5d4-4201-8dd5-a46ff90bbee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:41.033535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76f27a-f5d4-4201-8dd5-a46ff90bbee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:41.034871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76f27a-f5d4-4201-8dd5-a46ff90bbee1 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:41.053150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 515/1020] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:44.279979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92694-ee49-40b3-91ff-0e6440727606 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:44.280534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92694-ee49-40b3-91ff-0e6440727606 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:44.280534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be92694-ee49-40b3-91ff-0e6440727606 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:44.281893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 506/1021] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:44.287028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f0cf-2c31-46c5-827f-0fa855a9ad51 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:44.287475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f0cf-2c31-46c5-827f-0fa855a9ad51 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:44.287577 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f0cf-2c31-46c5-827f-0fa855a9ad51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:44.354891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-afb9f0cf-2c31-46c5-827f-0fa855a9ad51 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:44.356013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-afb9f0cf-2c31-46c5-827f-0fa855a9ad51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:44.374541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:44.454697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 516/1022] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:44.736612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be92694-ee49-40b3-91ff-0e6440727606 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:44.738142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2be92694-ee49-40b3-91ff-0e6440727606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:44.845575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be92694-ee49-40b3-91ff-0e6440727606 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:44.936337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00997d-af86-41e5-81b7-dd3a836246e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:44.936947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00997d-af86-41e5-81b7-dd3a836246e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:44.937226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00997d-af86-41e5-81b7-dd3a836246e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:44.937498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00997d-af86-41e5-81b7-dd3a836246e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:44.939388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c00997d-af86-41e5-81b7-dd3a836246e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:44.989101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:45.062417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00997d-af86-41e5-81b7-dd3a836246e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:45.064097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00997d-af86-41e5-81b7-dd3a836246e5 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:45.086382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 507/1023] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:49.373395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4aa7-e977-40e6-b265-1ee92c7960c5 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:49.373934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4aa7-e977-40e6-b265-1ee92c7960c5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:49.373934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4aa7-e977-40e6-b265-1ee92c7960c5 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:49.379167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 517/1024] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:17:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:49.381113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767d6b-90fa-4599-9c46-f82039736c91 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:49.381545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767d6b-90fa-4599-9c46-f82039736c91 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:49.381702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767d6b-90fa-4599-9c46-f82039736c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:49.449948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0c767d6b-90fa-4599-9c46-f82039736c91 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:49.451126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c767d6b-90fa-4599-9c46-f82039736c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:49.550189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:49.552486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 508/1025] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:49.838506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4aa7-e977-40e6-b265-1ee92c7960c5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:49.840402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bebd4aa7-e977-40e6-b265-1ee92c7960c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:49.940560 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4aa7-e977-40e6-b265-1ee92c7960c5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:49.987337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:50.012321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48934022-790f-49eb-b85b-ee7dcec16a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:50.012913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934022-790f-49eb-b85b-ee7dcec16a59 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:50.013186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934022-790f-49eb-b85b-ee7dcec16a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:50.013451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48934022-790f-49eb-b85b-ee7dcec16a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:50.015035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-48934022-790f-49eb-b85b-ee7dcec16a59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:50.123961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48934022-790f-49eb-b85b-ee7dcec16a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:50.125409 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48934022-790f-49eb-b85b-ee7dcec16a59 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:50.143891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 518/1026] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:55.157251 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b5279-df14-40d0-bdd8-cc11ec9e9202 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:55.158202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b5279-df14-40d0-bdd8-cc11ec9e9202 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:55.160066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b5279-df14-40d0-bdd8-cc11ec9e9202 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:55.161937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 509/1027] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:17:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:55.168157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041a2ba-f351-4dc8-a22f-a8f56eb11ab5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:55.168628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041a2ba-f351-4dc8-a22f-a8f56eb11ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:55.168725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041a2ba-f351-4dc8-a22f-a8f56eb11ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:55.275569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d041a2ba-f351-4dc8-a22f-a8f56eb11ab5 None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:55.277792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d041a2ba-f351-4dc8-a22f-a8f56eb11ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:55.297828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:55.394859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 519/1028] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:17:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:59.699592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071223bf-9234-4f45-b24a-207f40e7a510 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:59.700156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071223bf-9234-4f45-b24a-207f40e7a510 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:59.700156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071223bf-9234-4f45-b24a-207f40e7a510 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:59.701490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 510/1029] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:17:59.706569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80c070-371b-4880-878a-13b4ddf85580 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:59.706983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80c070-371b-4880-878a-13b4ddf85580 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:59.707271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80c070-371b-4880-878a-13b4ddf85580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:59.793495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9a80c070-371b-4880-878a-13b4ddf85580 None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:59.794778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9a80c070-371b-4880-878a-13b4ddf85580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:59.877365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:59.916036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 520/1030] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:17:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:01.015072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff324db-4a02-4316-9751-fcf7a1cdf707 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:01.015571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff324db-4a02-4316-9751-fcf7a1cdf707 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:01.015571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff324db-4a02-4316-9751-fcf7a1cdf707 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:01.016884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 511/1031] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:01.022162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa55c-0161-4ddd-bbfa-82097dba5448 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:01.022570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa55c-0161-4ddd-bbfa-82097dba5448 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:01.022712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa55c-0161-4ddd-bbfa-82097dba5448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:01.105985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9e8fa55c-0161-4ddd-bbfa-82097dba5448 None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:01.107190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9e8fa55c-0161-4ddd-bbfa-82097dba5448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:01.212010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 521/1032] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:04.817106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19780f3c-7599-4135-afb1-de6238c0efc2 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:04.817635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19780f3c-7599-4135-afb1-de6238c0efc2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:04.817635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19780f3c-7599-4135-afb1-de6238c0efc2 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:04.824548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 512/1033] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:04.825306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbba611-2b15-4351-9416-05b288b449f1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:04.826085 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbba611-2b15-4351-9416-05b288b449f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:04.826355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbba611-2b15-4351-9416-05b288b449f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:04.864666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:04.898537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2bbba611-2b15-4351-9416-05b288b449f1 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:04.899723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2bbba611-2b15-4351-9416-05b288b449f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:04.999282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 522/1034] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:05.301283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59836d-23cc-4f12-9070-2710696c31d9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:05.301824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59836d-23cc-4f12-9070-2710696c31d9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:05.302326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59836d-23cc-4f12-9070-2710696c31d9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:05.305002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 513/1035] 10.222.0.48 () {54 vars in 961 bytes} [Mon Oct 24 13:18:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:05.313472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee8b7-cc5d-455c-bfbd-ecd38411206b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:05.313763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee8b7-cc5d-455c-bfbd-ecd38411206b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:05.314033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ee8b7-cc5d-455c-bfbd-ecd38411206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:05.395754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-389ee8b7-cc5d-455c-bfbd-ecd38411206b None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:05.396965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-389ee8b7-cc5d-455c-bfbd-ecd38411206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:05.457383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:05.552660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 523/1036] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Oct 24 13:18:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:05.554902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59836d-23cc-4f12-9070-2710696c31d9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:05.556677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-df59836d-23cc-4f12-9070-2710696c31d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:05.734365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59836d-23cc-4f12-9070-2710696c31d9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:05.810771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:05.852193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba634791-bd16-466f-a5c8-b50256afcad5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:05.852865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba634791-bd16-466f-a5c8-b50256afcad5 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:05.853102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba634791-bd16-466f-a5c8-b50256afcad5 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:05.853346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba634791-bd16-466f-a5c8-b50256afcad5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:05.854920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ba634791-bd16-466f-a5c8-b50256afcad5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:05.955285 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba634791-bd16-466f-a5c8-b50256afcad5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:05.956907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba634791-bd16-466f-a5c8-b50256afcad5 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:05.977341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 514/1037] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:08.380209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d5e0a-d53f-487d-97fc-d9cbbf20fa3d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:08.380716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d5e0a-d53f-487d-97fc-d9cbbf20fa3d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:08.381025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d5e0a-d53f-487d-97fc-d9cbbf20fa3d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:08.382253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 524/1038] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:08.387287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cba68-b54e-4527-9d89-aca2c483beec None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:08.387685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cba68-b54e-4527-9d89-aca2c483beec None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:08.387866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cba68-b54e-4527-9d89-aca2c483beec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:08.457063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-422cba68-b54e-4527-9d89-aca2c483beec None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:08.458315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-422cba68-b54e-4527-9d89-aca2c483beec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:08.499740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:08.576367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 515/1039] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:08.868158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d5e0a-d53f-487d-97fc-d9cbbf20fa3d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:08.870826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-185d5e0a-d53f-487d-97fc-d9cbbf20fa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:09.063366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d5e0a-d53f-487d-97fc-d9cbbf20fa3d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:09.073885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:09.212729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1cb89-8467-4be2-90cc-e996fbae5a70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:09.213329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1cb89-8467-4be2-90cc-e996fbae5a70 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:09.213595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1cb89-8467-4be2-90cc-e996fbae5a70 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:09.213866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1cb89-8467-4be2-90cc-e996fbae5a70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:09.215788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e0e1cb89-8467-4be2-90cc-e996fbae5a70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:09.313521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1cb89-8467-4be2-90cc-e996fbae5a70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:09.315112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1cb89-8467-4be2-90cc-e996fbae5a70 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:09.332899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 525/1040] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:11.915327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd0e6d-edff-4c15-b0ea-eeb2ce3e8920 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:11.915904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd0e6d-edff-4c15-b0ea-eeb2ce3e8920 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:11.915904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd0e6d-edff-4c15-b0ea-eeb2ce3e8920 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:11.917412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 516/1041] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:11.922419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71999411-5089-467a-8b06-11da6c2d9d20 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:11.922812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71999411-5089-467a-8b06-11da6c2d9d20 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:11.922976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71999411-5089-467a-8b06-11da6c2d9d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:11.938281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:11.992334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-71999411-5089-467a-8b06-11da6c2d9d20 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:11.993663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-71999411-5089-467a-8b06-11da6c2d9d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:12.101546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 526/1042] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:12.395287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bedda9-42ae-40bc-a1f2-6ea59844b39a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:12.395565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bedda9-42ae-40bc-a1f2-6ea59844b39a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:12.395892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bedda9-42ae-40bc-a1f2-6ea59844b39a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:12.397421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 517/1043] 10.222.0.48 () {54 vars in 961 bytes} [Mon Oct 24 13:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:12.412204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b109a18-69ea-4aaa-9b2e-119f54491ecd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:12.412204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b109a18-69ea-4aaa-9b2e-119f54491ecd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:12.412204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b109a18-69ea-4aaa-9b2e-119f54491ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:12.481336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8b109a18-69ea-4aaa-9b2e-119f54491ecd None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:12.482517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b109a18-69ea-4aaa-9b2e-119f54491ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:12.518613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:12.606067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 527/1044] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Oct 24 13:18:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:12.615244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bedda9-42ae-40bc-a1f2-6ea59844b39a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:12.618702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f7bedda9-42ae-40bc-a1f2-6ea59844b39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:12.773561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bedda9-42ae-40bc-a1f2-6ea59844b39a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:12.794578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:12.893350 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10b27c-fa5f-44a0-8373-2c4bc5af6cc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:12.893947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b27c-fa5f-44a0-8373-2c4bc5af6cc2 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:12.894217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b27c-fa5f-44a0-8373-2c4bc5af6cc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:12.894478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b27c-fa5f-44a0-8373-2c4bc5af6cc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:12.896038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6e10b27c-fa5f-44a0-8373-2c4bc5af6cc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:12.997359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10b27c-fa5f-44a0-8373-2c4bc5af6cc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:12.999788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10b27c-fa5f-44a0-8373-2c4bc5af6cc2 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:13.020601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 518/1045] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:13.045550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d292b6-5722-48d5-a8d6-06ac77006bbf None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:13.046023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d292b6-5722-48d5-a8d6-06ac77006bbf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:13.046146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d292b6-5722-48d5-a8d6-06ac77006bbf None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:13.047469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 528/1046] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:13.052882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf75c51-1e30-4bc7-8cbb-e2d44b9b7528 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:13.053694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf75c51-1e30-4bc7-8cbb-e2d44b9b7528 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:13.053977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf75c51-1e30-4bc7-8cbb-e2d44b9b7528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:13.086139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:13.125073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ddf75c51-1e30-4bc7-8cbb-e2d44b9b7528 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:13.126420 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ddf75c51-1e30-4bc7-8cbb-e2d44b9b7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:13.236499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 519/1047] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:13.578120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc33ae-205a-4fdc-8d61-364c7023e8be None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:13.578120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc33ae-205a-4fdc-8d61-364c7023e8be None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:13.578120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc33ae-205a-4fdc-8d61-364c7023e8be None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:13.578120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 529/1048] 10.222.0.48 () {54 vars in 963 bytes} [Mon Oct 24 13:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:13.583348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66713-1af8-48dc-8078-61f028ff2c01 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:13.583715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66713-1af8-48dc-8078-61f028ff2c01 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:13.584182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f66713-1af8-48dc-8078-61f028ff2c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:13.669802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-79f66713-1af8-48dc-8078-61f028ff2c01 None None] MFA Rules not processed for user `32fa3c94cad04287aedea87400a15591`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:13.671177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-79f66713-1af8-48dc-8078-61f028ff2c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:13.740425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:13.759323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 520/1049] 10.222.0.48 () {58 vars in 1060 bytes} [Mon Oct 24 13:18:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:13.767316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc33ae-205a-4fdc-8d61-364c7023e8be None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:13.768833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aacc33ae-205a-4fdc-8d61-364c7023e8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:13.878163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc33ae-205a-4fdc-8d61-364c7023e8be None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:13.919980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:13.955202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a877df-f37c-41f6-9c11-1f512a6da0d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:13.955716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a877df-f37c-41f6-9c11-1f512a6da0d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:13.956423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a877df-f37c-41f6-9c11-1f512a6da0d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:13.957045 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a877df-f37c-41f6-9c11-1f512a6da0d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:13.959207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-93a877df-f37c-41f6-9c11-1f512a6da0d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:14.075647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a877df-f37c-41f6-9c11-1f512a6da0d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:14.077427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a877df-f37c-41f6-9c11-1f512a6da0d4 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:14.105801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 530/1050] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:15.414855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e9945-5b3c-4316-9538-6f670d67b6d7 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:15.415392 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e9945-5b3c-4316-9538-6f670d67b6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:15.415392 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e9945-5b3c-4316-9538-6f670d67b6d7 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:15.416875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 521/1051] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:15.422077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e27e5-14cd-4936-91a1-8c830ac485d9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:15.422502 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e27e5-14cd-4936-91a1-8c830ac485d9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:15.422746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e27e5-14cd-4936-91a1-8c830ac485d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:15.491763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-234e27e5-14cd-4936-91a1-8c830ac485d9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:15.493236 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-234e27e5-14cd-4936-91a1-8c830ac485d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:15.515216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:15.595083 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 531/1052] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:15.888873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e9945-5b3c-4316-9538-6f670d67b6d7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:15.890392 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ef6e9945-5b3c-4316-9538-6f670d67b6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:16.039074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e9945-5b3c-4316-9538-6f670d67b6d7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:16.099581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:16.164658 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410f8a2-c1c1-4dc8-9efe-67e60d65639f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:16.165256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f8a2-c1c1-4dc8-9efe-67e60d65639f service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:16.165529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f8a2-c1c1-4dc8-9efe-67e60d65639f service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:16.165793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410f8a2-c1c1-4dc8-9efe-67e60d65639f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:16.167494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a410f8a2-c1c1-4dc8-9efe-67e60d65639f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:16.269843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410f8a2-c1c1-4dc8-9efe-67e60d65639f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:16.271647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410f8a2-c1c1-4dc8-9efe-67e60d65639f service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:16.290606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 522/1053] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:18.622876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de1192-f67c-4ece-ada4-86f0ff2b7372 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.625203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de1192-f67c-4ece-ada4-86f0ff2b7372 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.625203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de1192-f67c-4ece-ada4-86f0ff2b7372 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.625203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 532/1054] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:18.630137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c955987-6728-422d-9e16-fb2385953500 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.630557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c955987-6728-422d-9e16-fb2385953500 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.630683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c955987-6728-422d-9e16-fb2385953500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.701196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4c955987-6728-422d-9e16-fb2385953500 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:18.702407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4c955987-6728-422d-9e16-fb2385953500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:18.710307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:18.809212 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 523/1055] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:19.098426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de1192-f67c-4ece-ada4-86f0ff2b7372 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:19.100594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-14de1192-f67c-4ece-ada4-86f0ff2b7372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:19.253205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de1192-f67c-4ece-ada4-86f0ff2b7372 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:19.257760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:19.372710 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2a45f-7c2a-4e2d-ab75-d12063387cd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:19.374036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a45f-7c2a-4e2d-ab75-d12063387cd4 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.374313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a45f-7c2a-4e2d-ab75-d12063387cd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.374584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a45f-7c2a-4e2d-ab75-d12063387cd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.376160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9cb2a45f-7c2a-4e2d-ab75-d12063387cd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:19.473831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2a45f-7c2a-4e2d-ab75-d12063387cd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:19.476076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2a45f-7c2a-4e2d-ab75-d12063387cd4 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:19.495447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 533/1056] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:21.823881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf9abb-74a2-40b6-90d6-b46088077d33 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:21.824457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf9abb-74a2-40b6-90d6-b46088077d33 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:21.824557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf9abb-74a2-40b6-90d6-b46088077d33 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:21.825832 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 524/1057] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:21.831352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6af32-8e9a-408e-b2f3-2da76ac4c2ab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:21.831789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6af32-8e9a-408e-b2f3-2da76ac4c2ab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:21.831951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6af32-8e9a-408e-b2f3-2da76ac4c2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:21.845037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:21.902512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c8e6af32-8e9a-408e-b2f3-2da76ac4c2ab None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:21.904077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c8e6af32-8e9a-408e-b2f3-2da76ac4c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:22.016434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 534/1058] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:22.311091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf9abb-74a2-40b6-90d6-b46088077d33 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:22.313106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4bdf9abb-74a2-40b6-90d6-b46088077d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:22.470135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:22.481059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf9abb-74a2-40b6-90d6-b46088077d33 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:22.597271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f6a03-087d-46ee-8b2a-bfbaa31ef98b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:22.597788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f6a03-087d-46ee-8b2a-bfbaa31ef98b service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:22.598065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f6a03-087d-46ee-8b2a-bfbaa31ef98b service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:22.598337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f6a03-087d-46ee-8b2a-bfbaa31ef98b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:22.599928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c2f6a03-087d-46ee-8b2a-bfbaa31ef98b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:22.690955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f6a03-087d-46ee-8b2a-bfbaa31ef98b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:22.692655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f6a03-087d-46ee-8b2a-bfbaa31ef98b service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:22.703378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:22.711912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 525/1059] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:25.432449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2a4d-de80-437e-8fed-fbf7d4567759 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.433001 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2a4d-de80-437e-8fed-fbf7d4567759 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.433001 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2a4d-de80-437e-8fed-fbf7d4567759 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:25.434345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 535/1060] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:25.440928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eda53-4635-4d26-9b56-e3a62aae00f4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.441378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eda53-4635-4d26-9b56-e3a62aae00f4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.441597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eda53-4635-4d26-9b56-e3a62aae00f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:25.513249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-154eda53-4635-4d26-9b56-e3a62aae00f4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:25.514472 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-154eda53-4635-4d26-9b56-e3a62aae00f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:25.624756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 526/1061] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:25.924261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2a4d-de80-437e-8fed-fbf7d4567759 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:25.926114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-066e2a4d-de80-437e-8fed-fbf7d4567759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:26.014997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:26.076737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2a4d-de80-437e-8fed-fbf7d4567759 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.196591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564bda7-9823-4384-9c25-811a9e8c3cac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.197113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564bda7-9823-4384-9c25-811a9e8c3cac service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.197402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564bda7-9823-4384-9c25-811a9e8c3cac service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.197682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564bda7-9823-4384-9c25-811a9e8c3cac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.199218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6564bda7-9823-4384-9c25-811a9e8c3cac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:26.253066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:26.295671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564bda7-9823-4384-9c25-811a9e8c3cac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.297327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564bda7-9823-4384-9c25-811a9e8c3cac service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.317045 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 536/1062] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:28.491843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45b69-6c29-45cc-bb78-080354a78697 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:28.492405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45b69-6c29-45cc-bb78-080354a78697 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:28.492497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af45b69-6c29-45cc-bb78-080354a78697 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:28.493767 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 527/1063] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:18:28.498988 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a3e5-fd73-40c2-8f44-7e5f862ce74e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:28.499432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a3e5-fd73-40c2-8f44-7e5f862ce74e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:28.499569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009a3e5-fd73-40c2-8f44-7e5f862ce74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:28.565678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f009a3e5-fd73-40c2-8f44-7e5f862ce74e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:28.566865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f009a3e5-fd73-40c2-8f44-7e5f862ce74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:28.664629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 537/1064] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:28.954275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af45b69-6c29-45cc-bb78-080354a78697 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:28.955882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5af45b69-6c29-45cc-bb78-080354a78697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:29.031743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:29.103838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af45b69-6c29-45cc-bb78-080354a78697 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:29.217557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f345f41-3857-4e7e-933f-3bef837970d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:29.218067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f345f41-3857-4e7e-933f-3bef837970d6 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:29.218364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f345f41-3857-4e7e-933f-3bef837970d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:29.218631 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f345f41-3857-4e7e-933f-3bef837970d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:29.220201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9f345f41-3857-4e7e-933f-3bef837970d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:29.258908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:29.309219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f345f41-3857-4e7e-933f-3bef837970d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:29.310790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f345f41-3857-4e7e-933f-3bef837970d6 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:29.328613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 528/1065] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:29.990825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0640df-176d-4b15-a305-2ab258dc1f01 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:29.991398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0640df-176d-4b15-a305-2ab258dc1f01 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:29.991398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0640df-176d-4b15-a305-2ab258dc1f01 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:29.992736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 538/1066] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:18:29.998095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562149-37a8-4a56-80fe-6342a8fcb02b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:29.998507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562149-37a8-4a56-80fe-6342a8fcb02b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:29.998645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562149-37a8-4a56-80fe-6342a8fcb02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:30.065940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f1562149-37a8-4a56-80fe-6342a8fcb02b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:30.067120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f1562149-37a8-4a56-80fe-6342a8fcb02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:30.166440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 529/1067] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:30.466364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430b5c-2f31-4e38-8e05-356d795ad9f9 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:30.466668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430b5c-2f31-4e38-8e05-356d795ad9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:30.467027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430b5c-2f31-4e38-8e05-356d795ad9f9 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:30.468480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 539/1068] 10.222.0.48 () {54 vars in 962 bytes} [Mon Oct 24 13:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:30.473839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357c6d-46d8-4bb7-bd35-79bd08c89f0a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:30.474130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357c6d-46d8-4bb7-bd35-79bd08c89f0a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:30.474403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357c6d-46d8-4bb7-bd35-79bd08c89f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:30.492359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:30.551008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1b357c6d-46d8-4bb7-bd35-79bd08c89f0a None None] MFA Rules not processed for user `6ee2800cf7f744a8973d5c5ca3907af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:30.552136 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1b357c6d-46d8-4bb7-bd35-79bd08c89f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:30.629337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 530/1069] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Oct 24 13:18:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:30.637721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30430b5c-2f31-4e38-8e05-356d795ad9f9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:30.639230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-30430b5c-2f31-4e38-8e05-356d795ad9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:30.671432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:30.746865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30430b5c-2f31-4e38-8e05-356d795ad9f9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:30.820615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e935a87-c22a-48bf-af5d-021702bd5764 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:30.821164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935a87-c22a-48bf-af5d-021702bd5764 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:30.821444 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935a87-c22a-48bf-af5d-021702bd5764 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:30.821716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e935a87-c22a-48bf-af5d-021702bd5764 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:30.823293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0e935a87-c22a-48bf-af5d-021702bd5764 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:30.899894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:30.931543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e935a87-c22a-48bf-af5d-021702bd5764 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:30.932997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e935a87-c22a-48bf-af5d-021702bd5764 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:30.956736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 540/1070] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:31.616200 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2330-3cfd-4619-9968-077ae184d913 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:31.616773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2330-3cfd-4619-9968-077ae184d913 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:31.616773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f2330-3cfd-4619-9968-077ae184d913 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:31.618130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 531/1071] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:31.624229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221fbde-969b-469e-8505-cf5c81332ba3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:31.624795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221fbde-969b-469e-8505-cf5c81332ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:31.628401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221fbde-969b-469e-8505-cf5c81332ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:31.695636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3221fbde-969b-469e-8505-cf5c81332ba3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:31.696906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3221fbde-969b-469e-8505-cf5c81332ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:31.794613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 541/1072] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:31.823852 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15389dc-de61-43cc-a5fb-190f62bb3549 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:31.824133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15389dc-de61-43cc-a5fb-190f62bb3549 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:31.824609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15389dc-de61-43cc-a5fb-190f62bb3549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:31.892476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d15389dc-de61-43cc-a5fb-190f62bb3549 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:31.893668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d15389dc-de61-43cc-a5fb-190f62bb3549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:31.947814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:31.978675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 532/1073] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:18:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:32.021129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df1ac-9882-44b4-9bb7-3b553b046676 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:32.021464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df1ac-9882-44b4-9bb7-3b553b046676 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:32.021740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327df1ac-9882-44b4-9bb7-3b553b046676 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:32.023101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 542/1074] 10.222.0.48 () {54 vars in 943 bytes} [Mon Oct 24 13:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:32.029180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de247b9-7ed7-4a29-a813-3f5492772b4e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:32.029469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de247b9-7ed7-4a29-a813-3f5492772b4e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:32.029745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de247b9-7ed7-4a29-a813-3f5492772b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:32.086989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327df1ac-9882-44b4-9bb7-3b553b046676 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:32.092459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-327df1ac-9882-44b4-9bb7-3b553b046676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:32.108465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2de247b9-7ed7-4a29-a813-3f5492772b4e None None] MFA Rules not processed for user `035a080de9d54a56af29cc56b33a1ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:32.109777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2de247b9-7ed7-4a29-a813-3f5492772b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:32.135102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:32.218639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 533/1075] 10.222.0.48 () {58 vars in 1040 bytes} [Mon Oct 24 13:18:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:32.227960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de247b9-7ed7-4a29-a813-3f5492772b4e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:32.237524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de247b9-7ed7-4a29-a813-3f5492772b4e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:32.244251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327df1ac-9882-44b4-9bb7-3b553b046676 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:32.264162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:32.345596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2f6fb-f1c6-483a-8098-7f2686d15445 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:32.346190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2f6fb-f1c6-483a-8098-7f2686d15445 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:32.346469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2f6fb-f1c6-483a-8098-7f2686d15445 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:32.346735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2f6fb-f1c6-483a-8098-7f2686d15445 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:32.348835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-10c2f6fb-f1c6-483a-8098-7f2686d15445 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:32.356599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:32.358973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec50dd-65e7-437f-8a2f-e5e2310aeedd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:32.359693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec50dd-65e7-437f-8a2f-e5e2310aeedd service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:32.359983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec50dd-65e7-437f-8a2f-e5e2310aeedd service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:32.360259 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec50dd-65e7-437f-8a2f-e5e2310aeedd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:32.361959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bbec50dd-65e7-437f-8a2f-e5e2310aeedd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:32.433160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2f6fb-f1c6-483a-8098-7f2686d15445 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:32.434727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2f6fb-f1c6-483a-8098-7f2686d15445 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:32.442817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 534/1076] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:18:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:18:32.454722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec50dd-65e7-437f-8a2f-e5e2310aeedd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:32.456440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec50dd-65e7-437f-8a2f-e5e2310aeedd service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:32.475283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 543/1077] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:18:34.851622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9321ee4-e052-4474-9abf-de97f86c0ac8 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:34.852130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9321ee4-e052-4474-9abf-de97f86c0ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:34.852252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9321ee4-e052-4474-9abf-de97f86c0ac8 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:34.853737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 535/1078] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:34.859602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4238c-2819-4146-9d6f-781bc44d67db None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:34.860051 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4238c-2819-4146-9d6f-781bc44d67db None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:34.860163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4238c-2819-4146-9d6f-781bc44d67db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:34.934568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3ec4238c-2819-4146-9d6f-781bc44d67db None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:34.935786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3ec4238c-2819-4146-9d6f-781bc44d67db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:35.015404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:35.045448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 544/1079] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:35.342291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9321ee4-e052-4474-9abf-de97f86c0ac8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:35.343864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f9321ee4-e052-4474-9abf-de97f86c0ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:35.407009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:35.509793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9321ee4-e052-4474-9abf-de97f86c0ac8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:35.641810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc00d5-8654-4756-8aa3-f39b8a2b60a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:35.642409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc00d5-8654-4756-8aa3-f39b8a2b60a5 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.642688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc00d5-8654-4756-8aa3-f39b8a2b60a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.642958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc00d5-8654-4756-8aa3-f39b8a2b60a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.644702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-09fc00d5-8654-4756-8aa3-f39b8a2b60a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:35.667752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:35.745792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc00d5-8654-4756-8aa3-f39b8a2b60a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:35.747389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc00d5-8654-4756-8aa3-f39b8a2b60a5 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:35.766931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 536/1080] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:38.116846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a62a49-04f4-4fda-9d63-704b225805d3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:38.117331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a62a49-04f4-4fda-9d63-704b225805d3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:38.117434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a62a49-04f4-4fda-9d63-704b225805d3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:38.118734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 545/1081] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:38.124152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77a85e-70ef-4f9b-b2b5-b314ccfb1613 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:38.126986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77a85e-70ef-4f9b-b2b5-b314ccfb1613 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:38.127295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77a85e-70ef-4f9b-b2b5-b314ccfb1613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:38.200249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6f77a85e-70ef-4f9b-b2b5-b314ccfb1613 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:38.201502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6f77a85e-70ef-4f9b-b2b5-b314ccfb1613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:38.309266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:38.315368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 537/1082] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:38.609413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a62a49-04f4-4fda-9d63-704b225805d3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:38.611329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-34a62a49-04f4-4fda-9d63-704b225805d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:38.841031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a62a49-04f4-4fda-9d63-704b225805d3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:38.901015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:38.970727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499719c-07f8-4a7b-aced-58f5201b3f58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:38.971679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499719c-07f8-4a7b-aced-58f5201b3f58 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:38.972175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499719c-07f8-4a7b-aced-58f5201b3f58 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:38.972558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499719c-07f8-4a7b-aced-58f5201b3f58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:38.974166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d499719c-07f8-4a7b-aced-58f5201b3f58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:39.079942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499719c-07f8-4a7b-aced-58f5201b3f58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:39.082284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499719c-07f8-4a7b-aced-58f5201b3f58 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:39.108460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 546/1083] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:41.449849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ddc6d-22cc-4a97-b29c-72626fdbdb48 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:41.450383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ddc6d-22cc-4a97-b29c-72626fdbdb48 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:41.450383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ddc6d-22cc-4a97-b29c-72626fdbdb48 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:41.451982 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 538/1084] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:41.457812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5196-036b-48d5-8dc4-8ff3403c3ca8 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:41.458713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5196-036b-48d5-8dc4-8ff3403c3ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:41.459524 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5196-036b-48d5-8dc4-8ff3403c3ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:41.531682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ba0c5196-036b-48d5-8dc4-8ff3403c3ca8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:41.532916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ba0c5196-036b-48d5-8dc4-8ff3403c3ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:41.546443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:41.642417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 547/1085] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:41.930587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ddc6d-22cc-4a97-b29c-72626fdbdb48 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:41.932094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-480ddc6d-22cc-4a97-b29c-72626fdbdb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:42.090003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ddc6d-22cc-4a97-b29c-72626fdbdb48 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:42.170169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:42.218885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b2908-f710-46d1-9ea8-8a9fe503b312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:42.219494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b2908-f710-46d1-9ea8-8a9fe503b312 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:42.219775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b2908-f710-46d1-9ea8-8a9fe503b312 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:42.220048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b2908-f710-46d1-9ea8-8a9fe503b312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:42.221673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-158b2908-f710-46d1-9ea8-8a9fe503b312 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:42.324359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b2908-f710-46d1-9ea8-8a9fe503b312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:42.325957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b2908-f710-46d1-9ea8-8a9fe503b312 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:42.346473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 539/1086] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:51.640078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=38 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:52.344153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=64 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:56.368961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46577836-dba8-48dd-b1a0-102bea758621 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:56.369490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46577836-dba8-48dd-b1a0-102bea758621 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:56.369490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46577836-dba8-48dd-b1a0-102bea758621 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:56.370787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 548/1087] 10.222.0.48 () {54 vars in 918 bytes} [Mon Oct 24 13:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:56.375866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aced3-d4ed-4e74-917e-c0509368da6d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:56.376338 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aced3-d4ed-4e74-917e-c0509368da6d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:56.376446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aced3-d4ed-4e74-917e-c0509368da6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:56.450430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ba0aced3-d4ed-4e74-917e-c0509368da6d None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:56.451622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ba0aced3-d4ed-4e74-917e-c0509368da6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:56.557824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 540/1088] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Oct 24 13:18:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:56.571848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf400ca7-0cbb-472e-8e39-a375291a379f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:56.572144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf400ca7-0cbb-472e-8e39-a375291a379f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:56.572456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf400ca7-0cbb-472e-8e39-a375291a379f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:56.573807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 549/1089] 10.222.0.48 () {54 vars in 918 bytes} [Mon Oct 24 13:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:56.578496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707d6e3-d222-4d3d-a8b6-9f0b5fd87615 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:56.578776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707d6e3-d222-4d3d-a8b6-9f0b5fd87615 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:56.579048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707d6e3-d222-4d3d-a8b6-9f0b5fd87615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:56.653075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:56.655569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6707d6e3-d222-4d3d-a8b6-9f0b5fd87615 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:56.656815 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6707d6e3-d222-4d3d-a8b6-9f0b5fd87615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:56.773813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 541/1090] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Oct 24 13:18:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:59.326471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341b74e-6004-45dc-883f-53eff24bba79 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:59.327034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341b74e-6004-45dc-883f-53eff24bba79 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:59.327034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341b74e-6004-45dc-883f-53eff24bba79 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:59.328398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 550/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:59.333673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa0d0-fcc8-418c-99a5-026314c9ba9e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:59.334123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa0d0-fcc8-418c-99a5-026314c9ba9e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:59.334230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa0d0-fcc8-418c-99a5-026314c9ba9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:59.399396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e84aa0d0-fcc8-418c-99a5-026314c9ba9e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:59.400616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e84aa0d0-fcc8-418c-99a5-026314c9ba9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:59.454265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:59.496903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 542/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:59.792601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341b74e-6004-45dc-883f-53eff24bba79 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:59.794139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8341b74e-6004-45dc-883f-53eff24bba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:59.942752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341b74e-6004-45dc-883f-53eff24bba79 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:00.014938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:00.054225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f2b4e-9454-4ad6-a52c-9c47c47e1f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:00.054757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f2b4e-9454-4ad6-a52c-9c47c47e1f6f service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:00.055031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f2b4e-9454-4ad6-a52c-9c47c47e1f6f service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:00.055298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f2b4e-9454-4ad6-a52c-9c47c47e1f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:00.056885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-554f2b4e-9454-4ad6-a52c-9c47c47e1f6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:00.144204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f2b4e-9454-4ad6-a52c-9c47c47e1f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:00.145809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f2b4e-9454-4ad6-a52c-9c47c47e1f6f service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:00.164000 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 551/1093] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:01.292496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dac547-5056-4d90-a095-a93f691e738d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:01.293021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dac547-5056-4d90-a095-a93f691e738d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:01.293117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dac547-5056-4d90-a095-a93f691e738d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:01.294402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 543/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:01.299452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a2f6e-386f-459e-a1b0-8dc54f04e9f4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:01.299886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a2f6e-386f-459e-a1b0-8dc54f04e9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:01.299992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a2f6e-386f-459e-a1b0-8dc54f04e9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:01.376182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7a8a2f6e-386f-459e-a1b0-8dc54f04e9f4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:01.377319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7a8a2f6e-386f-459e-a1b0-8dc54f04e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:01.403573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:01.471602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 552/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:01.622443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dac547-5056-4d90-a095-a93f691e738d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:01.624048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e3dac547-5056-4d90-a095-a93f691e738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:01.766986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dac547-5056-4d90-a095-a93f691e738d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:01.806426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:01.878513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1c5c0-95c6-474c-a08e-26d79ca022d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:01.879109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1c5c0-95c6-474c-a08e-26d79ca022d3 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:01.879384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1c5c0-95c6-474c-a08e-26d79ca022d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:01.879655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1c5c0-95c6-474c-a08e-26d79ca022d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:01.881271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f1f1c5c0-95c6-474c-a08e-26d79ca022d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:01.971205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1c5c0-95c6-474c-a08e-26d79ca022d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:01.972827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1c5c0-95c6-474c-a08e-26d79ca022d3 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:01.991218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 544/1096] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:03.097773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef908e-cdb9-4a8d-ac24-d8c870689022 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:03.098281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef908e-cdb9-4a8d-ac24-d8c870689022 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:03.098384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef908e-cdb9-4a8d-ac24-d8c870689022 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:03.099660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 553/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:03.105005 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650c93-39d3-45d8-aee2-68e9bab4f8ed None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:03.112811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650c93-39d3-45d8-aee2-68e9bab4f8ed None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:03.112811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c650c93-39d3-45d8-aee2-68e9bab4f8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:03.159382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:03.181445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7c650c93-39d3-45d8-aee2-68e9bab4f8ed None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:03.182703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c650c93-39d3-45d8-aee2-68e9bab4f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:03.277378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 545/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:03.427279 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef908e-cdb9-4a8d-ac24-d8c870689022 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:03.428828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0eef908e-cdb9-4a8d-ac24-d8c870689022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:03.572126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef908e-cdb9-4a8d-ac24-d8c870689022 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:03.584396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:03.685185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf407f-13b3-405c-9330-00068dce1d8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:03.685784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf407f-13b3-405c-9330-00068dce1d8e service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:03.686063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf407f-13b3-405c-9330-00068dce1d8e service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:03.686333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf407f-13b3-405c-9330-00068dce1d8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:03.687912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e7cf407f-13b3-405c-9330-00068dce1d8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:03.778040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf407f-13b3-405c-9330-00068dce1d8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:03.779610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf407f-13b3-405c-9330-00068dce1d8e service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:03.796807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 554/1099] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:04.901973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2a6b6-65c5-4921-a60c-44b6db5a14fe None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:04.902404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2a6b6-65c5-4921-a60c-44b6db5a14fe None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:04.902537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2a6b6-65c5-4921-a60c-44b6db5a14fe None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:04.903930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 546/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:19:04.908854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8bf8-6ece-4b60-b266-53c7fc0c93dd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:04.909301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8bf8-6ece-4b60-b266-53c7fc0c93dd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:04.909396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8bf8-6ece-4b60-b266-53c7fc0c93dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:04.929198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:04.984652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-debd8bf8-6ece-4b60-b266-53c7fc0c93dd None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:04.985872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-debd8bf8-6ece-4b60-b266-53c7fc0c93dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:05.080080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 555/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:05.229433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2a6b6-65c5-4921-a60c-44b6db5a14fe None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:05.230948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8ff2a6b6-65c5-4921-a60c-44b6db5a14fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:05.339907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:05.376071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2a6b6-65c5-4921-a60c-44b6db5a14fe None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.504817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd86a6-5d46-4441-80fe-2a1b9e782c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.504817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd86a6-5d46-4441-80fe-2a1b9e782c66 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.504817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd86a6-5d46-4441-80fe-2a1b9e782c66 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.504817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd86a6-5d46-4441-80fe-2a1b9e782c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.505413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9cdd86a6-5d46-4441-80fe-2a1b9e782c66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:05.574242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:05.587697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd86a6-5d46-4441-80fe-2a1b9e782c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.589792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd86a6-5d46-4441-80fe-2a1b9e782c66 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.608011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 547/1102] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:06.716894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d487-0d85-44cd-9e46-2d1e09ef448a None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.717504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d487-0d85-44cd-9e46-2d1e09ef448a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.717504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d487-0d85-44cd-9e46-2d1e09ef448a None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.718809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 556/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:06.723638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cca15e-73d9-45e2-afb2-dfc3ca694f7e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.724090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cca15e-73d9-45e2-afb2-dfc3ca694f7e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.724205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cca15e-73d9-45e2-afb2-dfc3ca694f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.799392 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-27cca15e-73d9-45e2-afb2-dfc3ca694f7e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:06.800539 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27cca15e-73d9-45e2-afb2-dfc3ca694f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:06.895541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 548/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:07.046558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d487-0d85-44cd-9e46-2d1e09ef448a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:07.048060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a2d9d487-0d85-44cd-9e46-2d1e09ef448a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:07.132671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:07.192252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d487-0d85-44cd-9e46-2d1e09ef448a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:07.302030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5654d9-e6b4-4ca6-a79d-1ca3297126f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:07.302623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5654d9-e6b4-4ca6-a79d-1ca3297126f1 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:07.302903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5654d9-e6b4-4ca6-a79d-1ca3297126f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:07.303830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5654d9-e6b4-4ca6-a79d-1ca3297126f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:07.305468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7e5654d9-e6b4-4ca6-a79d-1ca3297126f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:07.354220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:07.394228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5654d9-e6b4-4ca6-a79d-1ca3297126f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:07.395812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5654d9-e6b4-4ca6-a79d-1ca3297126f1 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:07.413812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 557/1105] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:08.495771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7f4d-87a2-4ecd-bd07-842eb946bb5e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:08.496428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7f4d-87a2-4ecd-bd07-842eb946bb5e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:08.496428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7f4d-87a2-4ecd-bd07-842eb946bb5e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:08.497710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 549/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:08.502604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800cde4-a58b-464a-a119-d019a6a093de None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:08.503033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800cde4-a58b-464a-a119-d019a6a093de None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:08.503142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800cde4-a58b-464a-a119-d019a6a093de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:08.578143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8800cde4-a58b-464a-a119-d019a6a093de None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:08.579339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8800cde4-a58b-464a-a119-d019a6a093de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:08.675382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 558/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:08.826467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f7f4d-87a2-4ecd-bd07-842eb946bb5e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:08.827908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-454f7f4d-87a2-4ecd-bd07-842eb946bb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:08.877305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:08.971708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f7f4d-87a2-4ecd-bd07-842eb946bb5e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:09.083207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36c0b2-d91a-46bd-80fa-e2e812351281 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:09.083725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36c0b2-d91a-46bd-80fa-e2e812351281 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:09.084000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36c0b2-d91a-46bd-80fa-e2e812351281 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:09.084300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36c0b2-d91a-46bd-80fa-e2e812351281 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:09.085884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8d36c0b2-d91a-46bd-80fa-e2e812351281 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:09.093233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:09.175932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36c0b2-d91a-46bd-80fa-e2e812351281 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:09.177576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36c0b2-d91a-46bd-80fa-e2e812351281 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:09.195130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 550/1108] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:10.285455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2fb4c-b847-40a6-ab1b-2cffb87d6c94 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:10.285873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2fb4c-b847-40a6-ab1b-2cffb87d6c94 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:10.286003 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2fb4c-b847-40a6-ab1b-2cffb87d6c94 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:10.287422 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 559/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:10.292626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddce27-df60-433c-8087-6a4f5967ae07 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:10.293060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddce27-df60-433c-8087-6a4f5967ae07 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:10.293170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddce27-df60-433c-8087-6a4f5967ae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:10.368164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d1ddce27-df60-433c-8087-6a4f5967ae07 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:10.369315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d1ddce27-df60-433c-8087-6a4f5967ae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:10.437705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:10.467220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 551/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:10.620450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2fb4c-b847-40a6-ab1b-2cffb87d6c94 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:10.620450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0ac2fb4c-b847-40a6-ab1b-2cffb87d6c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:10.638920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:10.765517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2fb4c-b847-40a6-ab1b-2cffb87d6c94 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:10.851327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:10.877337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6ab17-6b20-4eac-be18-8a07d427ea38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:10.877862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6ab17-6b20-4eac-be18-8a07d427ea38 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:10.878143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6ab17-6b20-4eac-be18-8a07d427ea38 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:10.878421 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6ab17-6b20-4eac-be18-8a07d427ea38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:10.879999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-77b6ab17-6b20-4eac-be18-8a07d427ea38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:10.968355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6ab17-6b20-4eac-be18-8a07d427ea38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:10.969930 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6ab17-6b20-4eac-be18-8a07d427ea38 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:10.987584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 560/1111] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:12.060026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1914d-9039-49e3-b74d-45b3004ddbbd None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:12.060632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1914d-9039-49e3-b74d-45b3004ddbbd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:12.060632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1914d-9039-49e3-b74d-45b3004ddbbd None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:12.061925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 552/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:12.066974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a7a12-1805-43e0-977c-2a0ec274ac8e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:12.067418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a7a12-1805-43e0-977c-2a0ec274ac8e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:12.067641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a7a12-1805-43e0-977c-2a0ec274ac8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:12.145125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2a6a7a12-1805-43e0-977c-2a0ec274ac8e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:12.146313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2a6a7a12-1805-43e0-977c-2a0ec274ac8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:12.187447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:12.241200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 561/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:12.389038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1914d-9039-49e3-b74d-45b3004ddbbd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:12.390489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c4d1914d-9039-49e3-b74d-45b3004ddbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:12.531790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1914d-9039-49e3-b74d-45b3004ddbbd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:12.585888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:12.644073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76e206-81d0-494c-bc57-3140f761d152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:12.644636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76e206-81d0-494c-bc57-3140f761d152 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:12.644913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76e206-81d0-494c-bc57-3140f761d152 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:12.645207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76e206-81d0-494c-bc57-3140f761d152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:12.647417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2e76e206-81d0-494c-bc57-3140f761d152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:12.736365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76e206-81d0-494c-bc57-3140f761d152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:12.737978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76e206-81d0-494c-bc57-3140f761d152 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:12.755439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 553/1114] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:13.857202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41436d-b312-4f7d-8d5b-8438afe18cd4 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:13.857779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41436d-b312-4f7d-8d5b-8438afe18cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:13.857779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41436d-b312-4f7d-8d5b-8438afe18cd4 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:13.859151 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 562/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:13.864163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4efa1-e903-416c-9bf2-9a8bab9ac052 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:13.864624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4efa1-e903-416c-9bf2-9a8bab9ac052 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:13.864727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4efa1-e903-416c-9bf2-9a8bab9ac052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:13.941073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0dd4efa1-e903-416c-9bf2-9a8bab9ac052 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:13.942323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0dd4efa1-e903-416c-9bf2-9a8bab9ac052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:13.944323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:14.038117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 554/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:14.187248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41436d-b312-4f7d-8d5b-8438afe18cd4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:14.188768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4c41436d-b312-4f7d-8d5b-8438afe18cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:14.333741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41436d-b312-4f7d-8d5b-8438afe18cd4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:14.360488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:14.444253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7dc39-668f-4204-942c-a59ed05788db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:14.444883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7dc39-668f-4204-942c-a59ed05788db service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:14.445162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7dc39-668f-4204-942c-a59ed05788db service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:14.445432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7dc39-668f-4204-942c-a59ed05788db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:14.447021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ffd7dc39-668f-4204-942c-a59ed05788db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:14.534136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7dc39-668f-4204-942c-a59ed05788db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:14.535716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7dc39-668f-4204-942c-a59ed05788db service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:14.553757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 563/1117] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:16.610299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e07c1-acdb-41ae-a637-4ce9f22acaf1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.610889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e07c1-acdb-41ae-a637-4ce9f22acaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.610889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e07c1-acdb-41ae-a637-4ce9f22acaf1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.612182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 555/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:16.618152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f86549-90f8-4ccb-84a5-701f46250929 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.618619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f86549-90f8-4ccb-84a5-701f46250929 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.618754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f86549-90f8-4ccb-84a5-701f46250929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.659455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:16.685201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c8f86549-90f8-4ccb-84a5-701f46250929 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:16.686451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c8f86549-90f8-4ccb-84a5-701f46250929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.780750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 564/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:17.062298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e07c1-acdb-41ae-a637-4ce9f22acaf1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.063849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ae5e07c1-acdb-41ae-a637-4ce9f22acaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:17.167886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e07c1-acdb-41ae-a637-4ce9f22acaf1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.199955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:17.243984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee060e78-7a1c-4924-bcba-ee423e61f304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.244606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee060e78-7a1c-4924-bcba-ee423e61f304 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.244884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee060e78-7a1c-4924-bcba-ee423e61f304 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.245155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee060e78-7a1c-4924-bcba-ee423e61f304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.246740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ee060e78-7a1c-4924-bcba-ee423e61f304 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:17.353415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee060e78-7a1c-4924-bcba-ee423e61f304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.354765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee060e78-7a1c-4924-bcba-ee423e61f304 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.373060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 556/1120] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:19.666868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499bad2-3612-4272-960d-c583d2c3145c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:19.667408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499bad2-3612-4272-960d-c583d2c3145c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:19.667408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499bad2-3612-4272-960d-c583d2c3145c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:19.668803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 565/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:19.674097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b165-9910-4120-9e80-1b465dd33129 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:19.674549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b165-9910-4120-9e80-1b465dd33129 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:19.674656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b165-9910-4120-9e80-1b465dd33129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:19.740606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1a45b165-9910-4120-9e80-1b465dd33129 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:19.741803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1a45b165-9910-4120-9e80-1b465dd33129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:19.754406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:19.837982 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 557/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:20.122753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499bad2-3612-4272-960d-c583d2c3145c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:20.124282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b499bad2-3612-4272-960d-c583d2c3145c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:20.226050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499bad2-3612-4272-960d-c583d2c3145c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:20.238511 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:20.301489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7719d2b-63f4-4a45-9d4a-885ce46f854f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:20.302080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7719d2b-63f4-4a45-9d4a-885ce46f854f service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:20.302356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7719d2b-63f4-4a45-9d4a-885ce46f854f service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:20.302625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7719d2b-63f4-4a45-9d4a-885ce46f854f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:20.304808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c7719d2b-63f4-4a45-9d4a-885ce46f854f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:20.410351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7719d2b-63f4-4a45-9d4a-885ce46f854f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:20.411764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7719d2b-63f4-4a45-9d4a-885ce46f854f service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:20.429882 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 566/1123] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:22.936635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457e5c-b354-499a-b4b3-360a960380d5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.937217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457e5c-b354-499a-b4b3-360a960380d5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.937217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24457e5c-b354-499a-b4b3-360a960380d5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.938490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 558/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:22.943978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295f96-5b1c-4608-a20a-670038854437 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.944518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295f96-5b1c-4608-a20a-670038854437 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.944630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295f96-5b1c-4608-a20a-670038854437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.992016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:23.009608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-26295f96-5b1c-4608-a20a-670038854437 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:23.010828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-26295f96-5b1c-4608-a20a-670038854437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:23.105535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 567/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:23.390506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457e5c-b354-499a-b4b3-360a960380d5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:23.392044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-24457e5c-b354-499a-b4b3-360a960380d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:23.493737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24457e5c-b354-499a-b4b3-360a960380d5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:23.532815 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:23.565288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51736bde-396c-421d-a64c-f20868bf9ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:23.565883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51736bde-396c-421d-a64c-f20868bf9ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:23.566161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51736bde-396c-421d-a64c-f20868bf9ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:23.566430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51736bde-396c-421d-a64c-f20868bf9ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:23.568013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-51736bde-396c-421d-a64c-f20868bf9ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:23.677425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51736bde-396c-421d-a64c-f20868bf9ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:23.678905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51736bde-396c-421d-a64c-f20868bf9ee7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:23.697228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 559/1126] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:25.967282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea0716-ec4d-46ea-842d-a19033a53d87 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.967826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea0716-ec4d-46ea-842d-a19033a53d87 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.967826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea0716-ec4d-46ea-842d-a19033a53d87 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.969226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 568/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:25.974545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b1cc-aa84-40f7-a4cd-1ffc38b71079 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.975590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b1cc-aa84-40f7-a4cd-1ffc38b71079 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.975768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b1cc-aa84-40f7-a4cd-1ffc38b71079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.039920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-51c7b1cc-aa84-40f7-a4cd-1ffc38b71079 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:26.041141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-51c7b1cc-aa84-40f7-a4cd-1ffc38b71079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:26.060857 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:26.136254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 560/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:26.418517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea0716-ec4d-46ea-842d-a19033a53d87 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:26.420056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6eea0716-ec4d-46ea-842d-a19033a53d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:26.523132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea0716-ec4d-46ea-842d-a19033a53d87 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:26.543199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:26.598332 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e3066-3814-404a-83a9-3f8984668595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:26.598922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3066-3814-404a-83a9-3f8984668595 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:26.599191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3066-3814-404a-83a9-3f8984668595 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:26.599453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e3066-3814-404a-83a9-3f8984668595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.601065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6d3e3066-3814-404a-83a9-3f8984668595 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:26.712011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e3066-3814-404a-83a9-3f8984668595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:26.713852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e3066-3814-404a-83a9-3f8984668595 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:26.731690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 569/1129] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:28.921882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12494-5dc1-413e-8963-81e64b5eaa0d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:28.922427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12494-5dc1-413e-8963-81e64b5eaa0d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:28.922427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12494-5dc1-413e-8963-81e64b5eaa0d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:28.923748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 561/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:28.929329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e4b94-86d7-4ad7-b227-b383b64f11ff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:28.929740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e4b94-86d7-4ad7-b227-b383b64f11ff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:28.929862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e4b94-86d7-4ad7-b227-b383b64f11ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:28.992049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:28.994625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9b0e4b94-86d7-4ad7-b227-b383b64f11ff None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:28.995806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9b0e4b94-86d7-4ad7-b227-b383b64f11ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:29.092494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 570/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:29.371478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca12494-5dc1-413e-8963-81e64b5eaa0d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:29.373080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eca12494-5dc1-413e-8963-81e64b5eaa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:29.475348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca12494-5dc1-413e-8963-81e64b5eaa0d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:29.523566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:29.547052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4d4d-8a48-4b4e-b1f5-a8145c9c6b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:29.552007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4d4d-8a48-4b4e-b1f5-a8145c9c6b44 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:29.552007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4d4d-8a48-4b4e-b1f5-a8145c9c6b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:29.552007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4d4d-8a48-4b4e-b1f5-a8145c9c6b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:29.552007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c5bb4d4d-8a48-4b4e-b1f5-a8145c9c6b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:29.658941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb4d4d-8a48-4b4e-b1f5-a8145c9c6b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:29.660385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb4d4d-8a48-4b4e-b1f5-a8145c9c6b44 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:29.679372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 562/1132] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:31.895100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a78c2-5d2a-4638-82b6-b64608a6f453 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:31.895640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a78c2-5d2a-4638-82b6-b64608a6f453 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:31.895640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a78c2-5d2a-4638-82b6-b64608a6f453 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:31.897060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 571/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:31.902286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d42697-befd-41ec-a587-29ece6418d14 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:31.902734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d42697-befd-41ec-a587-29ece6418d14 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:31.902840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d42697-befd-41ec-a587-29ece6418d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:31.967477 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-69d42697-befd-41ec-a587-29ece6418d14 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:31.968705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-69d42697-befd-41ec-a587-29ece6418d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:31.998516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:32.064490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 563/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:32.350842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a78c2-5d2a-4638-82b6-b64608a6f453 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:32.352410 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-969a78c2-5d2a-4638-82b6-b64608a6f453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:32.452644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a78c2-5d2a-4638-82b6-b64608a6f453 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:32.481252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:32.524914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be69d4-41a2-4706-895c-89e56bf2a32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:32.525505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be69d4-41a2-4706-895c-89e56bf2a32b service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:32.525782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be69d4-41a2-4706-895c-89e56bf2a32b service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:32.526048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be69d4-41a2-4706-895c-89e56bf2a32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:32.527621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-64be69d4-41a2-4706-895c-89e56bf2a32b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:32.634296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be69d4-41a2-4706-895c-89e56bf2a32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:32.635644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be69d4-41a2-4706-895c-89e56bf2a32b service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:32.654384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 572/1135] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:34.875093 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9b12-bbf4-4038-893a-a0b4dfd6a21b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:34.875638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9b12-bbf4-4038-893a-a0b4dfd6a21b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:34.875638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9b12-bbf4-4038-893a-a0b4dfd6a21b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:34.876991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 564/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:34.882168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be8408-bedb-4225-8f8e-a4fac055a3b5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:34.882591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be8408-bedb-4225-8f8e-a4fac055a3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:34.882707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be8408-bedb-4225-8f8e-a4fac055a3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:34.947471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-97be8408-bedb-4225-8f8e-a4fac055a3b5 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:34.948692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97be8408-bedb-4225-8f8e-a4fac055a3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:34.956855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:35.045678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 573/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:35.330984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b9b12-bbf4-4038-893a-a0b4dfd6a21b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:35.332577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-667b9b12-bbf4-4038-893a-a0b4dfd6a21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:35.433742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b9b12-bbf4-4038-893a-a0b4dfd6a21b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:35.488796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:35.506429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d19ffc-e6b0-49b9-b3a4-70a5278931ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:35.507024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d19ffc-e6b0-49b9-b3a4-70a5278931ce service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:35.511870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d19ffc-e6b0-49b9-b3a4-70a5278931ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:35.511870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d19ffc-e6b0-49b9-b3a4-70a5278931ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:35.511870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b6d19ffc-e6b0-49b9-b3a4-70a5278931ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:35.619085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d19ffc-e6b0-49b9-b3a4-70a5278931ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:35.620541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d19ffc-e6b0-49b9-b3a4-70a5278931ce service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:35.638390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 565/1138] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:37.837825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87a9bc-d1c5-4264-8762-584e015c4540 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:37.838640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87a9bc-d1c5-4264-8762-584e015c4540 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:37.838640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87a9bc-d1c5-4264-8762-584e015c4540 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:37.839877 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 574/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:37.845197 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631ffcc-39e4-4564-95ef-8cde0a9ea08f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:37.845639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631ffcc-39e4-4564-95ef-8cde0a9ea08f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:37.845748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631ffcc-39e4-4564-95ef-8cde0a9ea08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:37.911077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d631ffcc-39e4-4564-95ef-8cde0a9ea08f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:37.912879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d631ffcc-39e4-4564-95ef-8cde0a9ea08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:37.950349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:38.010763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 566/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:38.297513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87a9bc-d1c5-4264-8762-584e015c4540 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:38.299372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3d87a9bc-d1c5-4264-8762-584e015c4540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:38.400981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87a9bc-d1c5-4264-8762-584e015c4540 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:38.438553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:38.473508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be5988-cc6f-49f2-8d89-7741d94e2368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:38.474093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be5988-cc6f-49f2-8d89-7741d94e2368 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:38.474939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be5988-cc6f-49f2-8d89-7741d94e2368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:38.475212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be5988-cc6f-49f2-8d89-7741d94e2368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:38.476794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a5be5988-cc6f-49f2-8d89-7741d94e2368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:38.581269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be5988-cc6f-49f2-8d89-7741d94e2368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:38.582672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be5988-cc6f-49f2-8d89-7741d94e2368 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:38.602665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 575/1141] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:40.771055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae71a32-1f0a-44ec-82be-3c4c06a2c928 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:40.772306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae71a32-1f0a-44ec-82be-3c4c06a2c928 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:40.772306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae71a32-1f0a-44ec-82be-3c4c06a2c928 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:40.773510 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 567/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:19:40.778933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fde70-454c-47c3-ac5a-ea1e7388c77e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:40.779383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fde70-454c-47c3-ac5a-ea1e7388c77e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:40.779487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fde70-454c-47c3-ac5a-ea1e7388c77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:40.844739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7b8fde70-454c-47c3-ac5a-ea1e7388c77e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:40.845953 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7b8fde70-454c-47c3-ac5a-ea1e7388c77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:40.864335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:40.941165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 576/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:41.227833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae71a32-1f0a-44ec-82be-3c4c06a2c928 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.229478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6ae71a32-1f0a-44ec-82be-3c4c06a2c928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:41.332033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae71a32-1f0a-44ec-82be-3c4c06a2c928 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.398164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:41.404460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2b1cd-9a86-4e0e-939a-5df51fc04c26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.405055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2b1cd-9a86-4e0e-939a-5df51fc04c26 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.405331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2b1cd-9a86-4e0e-939a-5df51fc04c26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.405599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2b1cd-9a86-4e0e-939a-5df51fc04c26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.407179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9ae2b1cd-9a86-4e0e-939a-5df51fc04c26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:41.514949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2b1cd-9a86-4e0e-939a-5df51fc04c26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.516390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2b1cd-9a86-4e0e-939a-5df51fc04c26 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.533718 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 568/1144] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:43.682593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cae092-4967-4359-aae9-38b8e008e4d1 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:43.683176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cae092-4967-4359-aae9-38b8e008e4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:43.683176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cae092-4967-4359-aae9-38b8e008e4d1 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:43.684844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 577/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:19:43.690164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e68a7-189a-4978-a705-0a767dd0e93d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:43.690581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e68a7-189a-4978-a705-0a767dd0e93d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:43.690694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e68a7-189a-4978-a705-0a767dd0e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:43.756586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f63e68a7-189a-4978-a705-0a767dd0e93d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:43.757773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f63e68a7-189a-4978-a705-0a767dd0e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:43.805149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:43.853854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 569/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:44.146408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cae092-4967-4359-aae9-38b8e008e4d1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:44.147935 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-29cae092-4967-4359-aae9-38b8e008e4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:44.249211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cae092-4967-4359-aae9-38b8e008e4d1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:44.296573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:44.323004 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5b679-7b19-4ddf-8d50-979a1428f992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:44.323604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b679-7b19-4ddf-8d50-979a1428f992 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:44.323882 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b679-7b19-4ddf-8d50-979a1428f992 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:44.324152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b679-7b19-4ddf-8d50-979a1428f992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:44.325784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-48a5b679-7b19-4ddf-8d50-979a1428f992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:44.433339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5b679-7b19-4ddf-8d50-979a1428f992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:44.434797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5b679-7b19-4ddf-8d50-979a1428f992 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:44.452520 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 578/1147] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:53.852263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=22 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:54.450266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=66 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:04.326695 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768014fd-9bc1-4af9-9a00-6287ec41edac None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.327155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768014fd-9bc1-4af9-9a00-6287ec41edac None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.327261 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768014fd-9bc1-4af9-9a00-6287ec41edac None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.328611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 570/1148] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:20:04.333766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c262bca-8ef1-401f-a740-e9d9ddaabe85 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.334165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c262bca-8ef1-401f-a740-e9d9ddaabe85 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.334309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c262bca-8ef1-401f-a740-e9d9ddaabe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.399976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1c262bca-8ef1-401f-a740-e9d9ddaabe85 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:04.401203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1c262bca-8ef1-401f-a740-e9d9ddaabe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.494694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 579/1149] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:04.788821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e0d6-4166-4c74-875d-4c7f791a6e39 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.791727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e0d6-4166-4c74-875d-4c7f791a6e39 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.791727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e0d6-4166-4c74-875d-4c7f791a6e39 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.791727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 571/1150] 10.222.0.48 () {54 vars in 962 bytes} [Mon Oct 24 13:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:20:04.795636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a73d3-ed02-4c5e-9536-6155977408a7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.795921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a73d3-ed02-4c5e-9536-6155977408a7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.796191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a73d3-ed02-4c5e-9536-6155977408a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.872799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7d9a73d3-ed02-4c5e-9536-6155977408a7 None None] MFA Rules not processed for user `6ee2800cf7f744a8973d5c5ca3907af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:04.874032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7d9a73d3-ed02-4c5e-9536-6155977408a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.895036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:04.950434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 580/1151] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Oct 24 13:20:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:04.958373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0e0d6-4166-4c74-875d-4c7f791a6e39 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.959872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-46e0e0d6-4166-4c74-875d-4c7f791a6e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:05.061932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0e0d6-4166-4c74-875d-4c7f791a6e39 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.135384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d4d81-54e3-41c0-92d2-8bcff92a2f3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.135993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d4d81-54e3-41c0-92d2-8bcff92a2f3e service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.136250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d4d81-54e3-41c0-92d2-8bcff92a2f3e service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.136548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d4d81-54e3-41c0-92d2-8bcff92a2f3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.138151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-864d4d81-54e3-41c0-92d2-8bcff92a2f3e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:05.190901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:05.244416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d4d81-54e3-41c0-92d2-8bcff92a2f3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.245818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d4d81-54e3-41c0-92d2-8bcff92a2f3e service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.264243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 572/1152] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:07.508098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42121e-e138-4acb-ab94-fa5faefa8163 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:07.508725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42121e-e138-4acb-ab94-fa5faefa8163 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:07.508725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42121e-e138-4acb-ab94-fa5faefa8163 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:07.510057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 581/1153] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:20:07.515357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e62d15-7b79-47e0-9aff-9d4331ace1a7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:07.515774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e62d15-7b79-47e0-9aff-9d4331ace1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:07.515902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e62d15-7b79-47e0-9aff-9d4331ace1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:07.580785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-43e62d15-7b79-47e0-9aff-9d4331ace1a7 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:07.581992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-43e62d15-7b79-47e0-9aff-9d4331ace1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:07.678559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 573/1154] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:07.952300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42121e-e138-4acb-ab94-fa5faefa8163 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:07.953844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0c42121e-e138-4acb-ab94-fa5faefa8163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:08.076158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42121e-e138-4acb-ab94-fa5faefa8163 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.130859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:08.174135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f263a-2db7-4ad1-b2f8-61330cc6d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.174728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f263a-2db7-4ad1-b2f8-61330cc6d555 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.175006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f263a-2db7-4ad1-b2f8-61330cc6d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.175274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f263a-2db7-4ad1-b2f8-61330cc6d555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.175903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f263a-2db7-4ad1-b2f8-61330cc6d555 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.177033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-484f263a-2db7-4ad1-b2f8-61330cc6d555 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:20:08.177033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:20:08.177033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:20:08.178170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f263a-2db7-4ad1-b2f8-61330cc6d555 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.197106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 582/1155] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Oct 24 13:20:07 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:10.347021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9877e5-bdf4-45f3-a104-5f3f75456b48 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.347467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9877e5-bdf4-45f3-a104-5f3f75456b48 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.347578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9877e5-bdf4-45f3-a104-5f3f75456b48 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.349372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 574/1156] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:20:10.354668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d43db8-8fb8-4d10-b616-323bd77fae80 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.355086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d43db8-8fb8-4d10-b616-323bd77fae80 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.355196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d43db8-8fb8-4d10-b616-323bd77fae80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.419683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-89d43db8-8fb8-4d10-b616-323bd77fae80 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:10.420894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-89d43db8-8fb8-4d10-b616-323bd77fae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:10.516516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 583/1157] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:10.809532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9877e5-bdf4-45f3-a104-5f3f75456b48 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.811033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6d9877e5-bdf4-45f3-a104-5f3f75456b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:10.821484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:10.954149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9877e5-bdf4-45f3-a104-5f3f75456b48 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:11.034505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:11.068104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a498e5-cdda-4d9f-9067-0f45c3cda5cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:11.068661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a498e5-cdda-4d9f-9067-0f45c3cda5cf service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.068939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a498e5-cdda-4d9f-9067-0f45c3cda5cf service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.069210 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a498e5-cdda-4d9f-9067-0f45c3cda5cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.070831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-92a498e5-cdda-4d9f-9067-0f45c3cda5cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:11.159506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a498e5-cdda-4d9f-9067-0f45c3cda5cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:11.161611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a498e5-cdda-4d9f-9067-0f45c3cda5cf service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:11.179606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 575/1158] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:13.501629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c033-4d0e-46d3-91b7-32b1214108c8 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.502202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c033-4d0e-46d3-91b7-32b1214108c8 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.502202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40c033-4d0e-46d3-91b7-32b1214108c8 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.503529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 584/1159] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:20:13.508800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303bab7-0d1d-4217-a40e-c0432e71e042 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.509244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303bab7-0d1d-4217-a40e-c0432e71e042 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.509359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303bab7-0d1d-4217-a40e-c0432e71e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.574650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e303bab7-0d1d-4217-a40e-c0432e71e042 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:13.575827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e303bab7-0d1d-4217-a40e-c0432e71e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:13.621378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:13.674685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 576/1160] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:13.963334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40c033-4d0e-46d3-91b7-32b1214108c8 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.964903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fa40c033-4d0e-46d3-91b7-32b1214108c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:13.988553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:14.067458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40c033-4d0e-46d3-91b7-32b1214108c8 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:14.140742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90cbe2-eb28-41ec-9890-6c1ff0109e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:14.141332 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cbe2-eb28-41ec-9890-6c1ff0109e8d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:14.141609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cbe2-eb28-41ec-9890-6c1ff0109e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:14.141873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90cbe2-eb28-41ec-9890-6c1ff0109e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:14.143432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ff90cbe2-eb28-41ec-9890-6c1ff0109e8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:14.219104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:14.252440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90cbe2-eb28-41ec-9890-6c1ff0109e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:14.253876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90cbe2-eb28-41ec-9890-6c1ff0109e8d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:14.272683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 585/1161] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:16.477315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e298f0-44ef-4579-8d45-b4433f1ae69b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:16.477921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e298f0-44ef-4579-8d45-b4433f1ae69b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:16.477921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e298f0-44ef-4579-8d45-b4433f1ae69b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:16.479241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 577/1162] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:20:16.484471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a778f29-c859-4413-87bb-b7695dbedf37 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:16.484853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a778f29-c859-4413-87bb-b7695dbedf37 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:16.485022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a778f29-c859-4413-87bb-b7695dbedf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:16.549598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0a778f29-c859-4413-87bb-b7695dbedf37 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:16.550787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0a778f29-c859-4413-87bb-b7695dbedf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:16.646940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 586/1163] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:16.940560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e298f0-44ef-4579-8d45-b4433f1ae69b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:16.942100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-30e298f0-44ef-4579-8d45-b4433f1ae69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:17.044418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e298f0-44ef-4579-8d45-b4433f1ae69b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:17.117727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:17.119330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa4b07-e2a1-485e-8c3b-e9c80eef903a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:17.120026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa4b07-e2a1-485e-8c3b-e9c80eef903a service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:17.120376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa4b07-e2a1-485e-8c3b-e9c80eef903a service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:17.120694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa4b07-e2a1-485e-8c3b-e9c80eef903a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:17.122421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d6fa4b07-e2a1-485e-8c3b-e9c80eef903a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:17.233579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa4b07-e2a1-485e-8c3b-e9c80eef903a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:17.234929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa4b07-e2a1-485e-8c3b-e9c80eef903a service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:17.252442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 578/1164] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:19.480155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450116c-1334-4182-b986-78d07b7cf722 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:19.480778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450116c-1334-4182-b986-78d07b7cf722 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:19.480778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450116c-1334-4182-b986-78d07b7cf722 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:19.482056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 587/1165] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:20:19.487367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64c65-ffe6-4d5a-9c44-aaa5823369ef None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:19.487799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64c65-ffe6-4d5a-9c44-aaa5823369ef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:19.487902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f64c65-ffe6-4d5a-9c44-aaa5823369ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:19.553798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-32f64c65-ffe6-4d5a-9c44-aaa5823369ef None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:19.554996 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-32f64c65-ffe6-4d5a-9c44-aaa5823369ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:19.607746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:19.652020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 579/1166] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:19.943345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450116c-1334-4182-b986-78d07b7cf722 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:19.944884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6450116c-1334-4182-b986-78d07b7cf722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:19.979945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:20.045926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450116c-1334-4182-b986-78d07b7cf722 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:20.118253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bede53-84e0-4701-9737-5eefac951553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:20.118837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bede53-84e0-4701-9737-5eefac951553 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:20.119112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bede53-84e0-4701-9737-5eefac951553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:20.119376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bede53-84e0-4701-9737-5eefac951553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:20.120965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e1bede53-84e0-4701-9737-5eefac951553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:20.204498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:20.233268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bede53-84e0-4701-9737-5eefac951553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:20.234690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bede53-84e0-4701-9737-5eefac951553 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:20.253278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 588/1167] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:22.814585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22812368-411f-4074-a166-fa0981e5cf18 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:22.815155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22812368-411f-4074-a166-fa0981e5cf18 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:22.815155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22812368-411f-4074-a166-fa0981e5cf18 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:22.816498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 580/1168] 10.222.0.48 () {54 vars in 927 bytes} [Mon Oct 24 13:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:20:22.821850 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417efaf-7f31-4260-a12b-22b72a231938 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:22.822256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417efaf-7f31-4260-a12b-22b72a231938 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:22.822428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417efaf-7f31-4260-a12b-22b72a231938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:22.887464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3417efaf-7f31-4260-a12b-22b72a231938 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:22.888697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3417efaf-7f31-4260-a12b-22b72a231938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:22.983351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 589/1169] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Oct 24 13:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:23.272840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22812368-411f-4074-a166-fa0981e5cf18 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:23.274389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-22812368-411f-4074-a166-fa0981e5cf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:23.376209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22812368-411f-4074-a166-fa0981e5cf18 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:23.448527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111b9e1-7798-4683-9ce8-9de8c500c6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:23.449117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b9e1-7798-4683-9ce8-9de8c500c6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:23.449398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b9e1-7798-4683-9ce8-9de8c500c6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:23.449667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b9e1-7798-4683-9ce8-9de8c500c6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:23.451236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b111b9e1-7798-4683-9ce8-9de8c500c6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:23.459632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:23.558716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111b9e1-7798-4683-9ce8-9de8c500c6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:23.560119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111b9e1-7798-4683-9ce8-9de8c500c6b3 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:23.578615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 581/1170] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:32.981870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=82 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:33.576398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=50 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:45.322723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7f657-0b2c-4352-9570-35322cc96bc0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:45.323316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7f657-0b2c-4352-9570-35322cc96bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:45.323316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7f657-0b2c-4352-9570-35322cc96bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:45.397531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-84a7f657-0b2c-4352-9570-35322cc96bc0 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:45.398714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-84a7f657-0b2c-4352-9570-35322cc96bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:45.493335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 590/1171] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:45.501224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4dc88-1fff-4c9d-ac69-1bfa362b2c6d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:45.501726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4dc88-1fff-4c9d-ac69-1bfa362b2c6d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:45.501831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4dc88-1fff-4c9d-ac69-1bfa362b2c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:45.577243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-90d4dc88-1fff-4c9d-ac69-1bfa362b2c6d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:45.578355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-90d4dc88-1fff-4c9d-ac69-1bfa362b2c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:45.675422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 582/1172] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:45.680022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7f657-0b2c-4352-9570-35322cc96bc0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:45.681315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-84a7f657-0b2c-4352-9570-35322cc96bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:45.692556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7f657-0b2c-4352-9570-35322cc96bc0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:45.753146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:45.787557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:45.788143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:45.788448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:45.788715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:45.789336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:21:45.789630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:45.790631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:21:45.790631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:21:45.790631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:21:45.791508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:45.802467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 591/1173] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:21:45.812407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4dc88-1fff-4c9d-ac69-1bfa362b2c6d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:45.822092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4dc88-1fff-4c9d-ac69-1bfa362b2c6d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:45.883811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:45.916150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:45.916773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:45.917048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:45.917319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:45.917937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:45.918935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:21:45.918935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Oct 24 13:21:45.918935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: Oct 24 13:21:45.920115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:46.022730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 583/1174] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:21:45 2022] POST /identity/v3/projects => generated 379 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:46.026799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:46.028865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:46.061417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108565-8921-495e-a934-7bec61dcd1ff admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:46.155349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:46.155947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:46.156223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:46.156528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:46.157143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:46.158143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:21:46.158143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:21:46.158143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:21:46.159355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:46.271976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 592/1175] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:21:46 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:46.276043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:46.277525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:46.337982 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bb9cc-798b-43ad-a7ce-64c5fb765c3f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:46.421110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:46.432229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:46.432858 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:46.433137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:46.433401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:46.434036 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:46.435622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:46.447862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 584/1176] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:46.451915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:46.453460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:46.465113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80d642-72d8-4694-bfce-339848e79bec admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:46.491466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:46.559519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883c32b-91b1-4a2e-838c-7a11d2b5cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:46.560047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883c32b-91b1-4a2e-838c-7a11d2b5cd63 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:46.560347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883c32b-91b1-4a2e-838c-7a11d2b5cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:46.560615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883c32b-91b1-4a2e-838c-7a11d2b5cd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:46.561241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883c32b-91b1-4a2e-838c-7a11d2b5cd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:46.562898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883c32b-91b1-4a2e-838c-7a11d2b5cd63 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:46.575358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 593/1177] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:46.579665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:46.581306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:46.615557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a1111-6493-401c-9017-b1d5eb0757a4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:46.708183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c329b-2414-499d-b8af-ad1b7405d522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:46.708974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c329b-2414-499d-b8af-ad1b7405d522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:46.709776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c329b-2414-499d-b8af-ad1b7405d522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:46.710049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c329b-2414-499d-b8af-ad1b7405d522 admin admin] PATH_INFO: `/v3/projects/8d42fc337e194c1c8e6e62d54bc6d50f/users/ce2cfdff4374401e88373d67c2e82428/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:46.733424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c329b-2414-499d-b8af-ad1b7405d522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d42fc337e194c1c8e6e62d54bc6d50f, user_id=ce2cfdff4374401e88373d67c2e82428, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:46.736094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c329b-2414-499d-b8af-ad1b7405d522 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:46.833010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 585/1178] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:21:46 2022] PUT /identity/v3/projects/8d42fc337e194c1c8e6e62d54bc6d50f/users/ce2cfdff4374401e88373d67c2e82428/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:46.844887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edbb59-fedd-4580-bad0-acc670de80a0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:46.845173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edbb59-fedd-4580-bad0-acc670de80a0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:46.845468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edbb59-fedd-4580-bad0-acc670de80a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:46.902152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-62edbb59-fedd-4580-bad0-acc670de80a0 None None] MFA Rules not processed for user `ce2cfdff4374401e88373d67c2e82428`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:46.903351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-62edbb59-fedd-4580-bad0-acc670de80a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:46.979608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 594/1179] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:21:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:21:46.992666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43bf40-58cf-40c6-8372-491fc5fea3ec None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:46.992969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43bf40-58cf-40c6-8372-491fc5fea3ec None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:46.993240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43bf40-58cf-40c6-8372-491fc5fea3ec None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:46.994594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 586/1180] 10.222.0.48 () {54 vars in 963 bytes} [Mon Oct 24 13:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:21:46.999539 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e01d21-501b-474c-b867-b3be528b2e01 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:46.999829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e01d21-501b-474c-b867-b3be528b2e01 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:47.000105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e01d21-501b-474c-b867-b3be528b2e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:47.031904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:47.075464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-75e01d21-501b-474c-b867-b3be528b2e01 None None] MFA Rules not processed for user `32fa3c94cad04287aedea87400a15591`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:47.076660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-75e01d21-501b-474c-b867-b3be528b2e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:47.151700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 595/1181] 10.222.0.48 () {58 vars in 1060 bytes} [Mon Oct 24 13:21:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:47.159719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43bf40-58cf-40c6-8372-491fc5fea3ec None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:47.161282 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd43bf40-58cf-40c6-8372-491fc5fea3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:47.210316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:47.262864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43bf40-58cf-40c6-8372-491fc5fea3ec None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:47.338370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:47.338972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:47.339248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:47.339516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:47.341154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:47.440512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:47.441934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:47.446032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:47.459979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 587/1182] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:47.504615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1398-78e2-4ac2-bf73-84b7e7ea95cf None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:47.505039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1398-78e2-4ac2-bf73-84b7e7ea95cf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:47.505175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1398-78e2-4ac2-bf73-84b7e7ea95cf None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:47.507052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 596/1183] 10.222.0.48 () {52 vars in 830 bytes} [Mon Oct 24 13:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:21:47.510506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:47.511997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:47.609425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a593d1d-b964-45a0-9f81-8a3b0fbb968f service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:47.701190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:47.701735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:47.702011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:47.702277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] PATH_INFO: `/v3/users/ce2cfdff4374401e88373d67c2e82428` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:47.715098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2cfdff4374401e88373d67c2e82428)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:47.716101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:21:47.716101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Oct 24 13:21:47.716101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: Oct 24 13:21:47.717345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:47.727476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:47.910165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2cfdff4374401e88373d67c2e82428'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:47.923887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2cfdff4374401e88373d67c2e82428'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:47.947354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2cfdff4374401e88373d67c2e82428'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:48.020689 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2cfdff4374401e88373d67c2e82428'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:48.021149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2cfdff4374401e88373d67c2e82428'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:48.027876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 588/1184] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:21:47 2022] DELETE /identity/v3/users/ce2cfdff4374401e88373d67c2e82428 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:48.032249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73bf8e-374a-4c88-b93f-d35455b05b60 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:48.036818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73bf8e-374a-4c88-b93f-d35455b05b60 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:48.036818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73bf8e-374a-4c88-b93f-d35455b05b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:48.089633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8e73bf8e-374a-4c88-b93f-d35455b05b60 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:48.090857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8e73bf8e-374a-4c88-b93f-d35455b05b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:48.164963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:48.176579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 597/1185] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:48.187399 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:48.188639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:48.288353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1c375-b88a-4d80-908b-5c5939b92281 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:48.341717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:48.360245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc3f5-dd08-4221-85ce-e0abeedd4c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:48.360876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc3f5-dd08-4221-85ce-e0abeedd4c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:48.361152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc3f5-dd08-4221-85ce-e0abeedd4c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:48.361421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc3f5-dd08-4221-85ce-e0abeedd4c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:48.363058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a6dfc3f5-dd08-4221-85ce-e0abeedd4c7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:48.472908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfc3f5-dd08-4221-85ce-e0abeedd4c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:48.474344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfc3f5-dd08-4221-85ce-e0abeedd4c7c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:48.492503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 589/1186] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:48.937623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73bf8e-374a-4c88-b93f-d35455b05b60 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:48.938860 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8e73bf8e-374a-4c88-b93f-d35455b05b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:48.951260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73bf8e-374a-4c88-b93f-d35455b05b60 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:49.048095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:49.048751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:49.049028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:49.049374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] PATH_INFO: `/v3/projects/8d42fc337e194c1c8e6e62d54bc6d50f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:49.059521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d42fc337e194c1c8e6e62d54bc6d50f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:49.060570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:21:49.060570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:21:49.060570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:21:49.061756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:49.161650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d42fc337e194c1c8e6e62d54bc6d50f', 'request_id': 'req-46fc3052-e966-438f-a3cb-964d244a06f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:49.237924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d42fc337e194c1c8e6e62d54bc6d50f', 'request_id': 'req-46fc3052-e966-438f-a3cb-964d244a06f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:49.253949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d42fc337e194c1c8e6e62d54bc6d50f', 'request_id': 'req-46fc3052-e966-438f-a3cb-964d244a06f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:49.277462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] The token cache is being invalidate because project 8d42fc337e194c1c8e6e62d54bc6d50f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:21:49.277785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:49.278283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-46fc3052-e966-438f-a3cb-964d244a06f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:49.279849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 598/1187] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:21:48 2022] DELETE /identity/v3/projects/8d42fc337e194c1c8e6e62d54bc6d50f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:58.490262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=62 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:59.276955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:23:17.708651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ffe7-e64e-4c31-ae19-b204ebbb0a36 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:17.709254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ffe7-e64e-4c31-ae19-b204ebbb0a36 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:17.709254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406ffe7-e64e-4c31-ae19-b204ebbb0a36 None None] PATH_INFO: `/v3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:17.710699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 590/1188] 10.222.0.59 () {54 vars in 925 bytes} [Mon Oct 24 13:23:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:23:17.717093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374adda-ea82-421a-866f-ad445d427d86 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:17.717540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374adda-ea82-421a-866f-ad445d427d86 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:17.717647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374adda-ea82-421a-866f-ad445d427d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:17.796792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d374adda-ea82-421a-866f-ad445d427d86 None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:17.798000 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d374adda-ea82-421a-866f-ad445d427d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:17.909798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 599/1189] 10.222.0.59 () {58 vars in 1015 bytes} [Mon Oct 24 13:23:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:23:19.402844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b811a-c7a1-433a-a9ea-2672bcd9e265 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:19.403397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b811a-c7a1-433a-a9ea-2672bcd9e265 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:19.403397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b811a-c7a1-433a-a9ea-2672bcd9e265 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:19.404752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 591/1190] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:23:19.409637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eb023-af01-4088-b40b-a7e8839d00e6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:19.410096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eb023-af01-4088-b40b-a7e8839d00e6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:19.410213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19eb023-af01-4088-b40b-a7e8839d00e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:19.468646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b19eb023-af01-4088-b40b-a7e8839d00e6 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:19.469843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b19eb023-af01-4088-b40b-a7e8839d00e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:19.488464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:23:19.545949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 600/1191] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:19.698767 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b811a-c7a1-433a-a9ea-2672bcd9e265 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:19.700961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-559b811a-c7a1-433a-a9ea-2672bcd9e265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:19.803686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b811a-c7a1-433a-a9ea-2672bcd9e265 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:19.809130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22606a7-8629-4d61-962f-7107b1e47e3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:19.809723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22606a7-8629-4d61-962f-7107b1e47e3c service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:19.809997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22606a7-8629-4d61-962f-7107b1e47e3c service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:19.810267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22606a7-8629-4d61-962f-7107b1e47e3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:19.818145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22606a7-8629-4d61-962f-7107b1e47e3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:19.819741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22606a7-8629-4d61-962f-7107b1e47e3c service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:19.829603 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 592/1192] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:20.885248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65260cf0-5cdf-473c-b7d7-5653d6a5cc9d None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:20.885795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65260cf0-5cdf-473c-b7d7-5653d6a5cc9d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:20.885795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65260cf0-5cdf-473c-b7d7-5653d6a5cc9d None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:20.887123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 601/1193] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:23:20.891865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b06b47-ff3e-42fb-90cd-7a1779f7ca6f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:20.892341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b06b47-ff3e-42fb-90cd-7a1779f7ca6f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:20.892451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b06b47-ff3e-42fb-90cd-7a1779f7ca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:20.912148 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f0b06b47-ff3e-42fb-90cd-7a1779f7ca6f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:20.913347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f0b06b47-ff3e-42fb-90cd-7a1779f7ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:20.920962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 593/1194] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:21.074059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65260cf0-5cdf-473c-b7d7-5653d6a5cc9d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:21.088675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65260cf0-5cdf-473c-b7d7-5653d6a5cc9d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:21.094108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137ec81-7df0-4028-a3e0-e0314da61496 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:21.094704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137ec81-7df0-4028-a3e0-e0314da61496 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:21.094986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137ec81-7df0-4028-a3e0-e0314da61496 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:21.095261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137ec81-7df0-4028-a3e0-e0314da61496 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:21.103380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137ec81-7df0-4028-a3e0-e0314da61496 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:21.105017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137ec81-7df0-4028-a3e0-e0314da61496 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:21.114304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 602/1195] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:25.084208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ffd5-e02f-4dc3-ab0e-08739e1a9dee None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:25.085011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ffd5-e02f-4dc3-ab0e-08739e1a9dee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:25.085104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ffd5-e02f-4dc3-ab0e-08739e1a9dee None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:25.086371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 594/1196] 10.222.0.48 () {54 vars in 961 bytes} [Mon Oct 24 13:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:23:25.091483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3004df-8e88-4e2a-a6ad-1a9aec5fc9c0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:25.091946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3004df-8e88-4e2a-a6ad-1a9aec5fc9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:25.092083 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3004df-8e88-4e2a-a6ad-1a9aec5fc9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:25.111668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-be3004df-8e88-4e2a-a6ad-1a9aec5fc9c0 None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:25.112889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-be3004df-8e88-4e2a-a6ad-1a9aec5fc9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:25.122084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 603/1197] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Oct 24 13:23:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:25.129486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9ffd5-e02f-4dc3-ab0e-08739e1a9dee None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:25.142856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9ffd5-e02f-4dc3-ab0e-08739e1a9dee None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:25.147652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b26bb-8336-4b9b-aef0-0fcf64bfc383 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:25.148225 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b26bb-8336-4b9b-aef0-0fcf64bfc383 service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:25.148530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b26bb-8336-4b9b-aef0-0fcf64bfc383 service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:25.148800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b26bb-8336-4b9b-aef0-0fcf64bfc383 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:25.156692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b26bb-8336-4b9b-aef0-0fcf64bfc383 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:25.158243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b26bb-8336-4b9b-aef0-0fcf64bfc383 service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:25.167972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 595/1198] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:32.541105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0157d2f-ad82-476c-b9a7-7c64e11e8b6a None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:32.541663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0157d2f-ad82-476c-b9a7-7c64e11e8b6a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:32.541663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0157d2f-ad82-476c-b9a7-7c64e11e8b6a None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:32.542972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 604/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:23:32.547774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e83a2-0106-48a9-907f-b52ce2118c5f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:32.548192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e83a2-0106-48a9-907f-b52ce2118c5f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:32.548368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e83a2-0106-48a9-907f-b52ce2118c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:32.567752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-222e83a2-0106-48a9-907f-b52ce2118c5f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:32.568970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-222e83a2-0106-48a9-907f-b52ce2118c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:32.577274 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 596/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:32.729522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0157d2f-ad82-476c-b9a7-7c64e11e8b6a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:32.743781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0157d2f-ad82-476c-b9a7-7c64e11e8b6a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:32.749700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a18df-e047-405a-a491-102534e01878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:32.750283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a18df-e047-405a-a491-102534e01878 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:32.750562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a18df-e047-405a-a491-102534e01878 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:32.750832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a18df-e047-405a-a491-102534e01878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:32.758803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a18df-e047-405a-a491-102534e01878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:32.761179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a18df-e047-405a-a491-102534e01878 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:32.770708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 605/1201] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:33.821686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab4280-57eb-4cde-ad11-f0c6efbbc434 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:33.822722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab4280-57eb-4cde-ad11-f0c6efbbc434 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:33.822722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab4280-57eb-4cde-ad11-f0c6efbbc434 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:33.824077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 597/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:23:33.828881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cf8d7-cd31-4e0b-93a5-380dbf466c40 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:33.829449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cf8d7-cd31-4e0b-93a5-380dbf466c40 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:33.829449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cf8d7-cd31-4e0b-93a5-380dbf466c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:33.849426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a83cf8d7-cd31-4e0b-93a5-380dbf466c40 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:33.850617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a83cf8d7-cd31-4e0b-93a5-380dbf466c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:33.858797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 606/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:23:34.009129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab4280-57eb-4cde-ad11-f0c6efbbc434 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:34.023532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab4280-57eb-4cde-ad11-f0c6efbbc434 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:34.029402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dbfe4-cac7-477a-8436-fa5ec6cf8aa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:34.029980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dbfe4-cac7-477a-8436-fa5ec6cf8aa4 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:34.030255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dbfe4-cac7-477a-8436-fa5ec6cf8aa4 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:34.030523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dbfe4-cac7-477a-8436-fa5ec6cf8aa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:34.038361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dbfe4-cac7-477a-8436-fa5ec6cf8aa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:34.039920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dbfe4-cac7-477a-8436-fa5ec6cf8aa4 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:34.048943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 598/1204] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:38.211609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6962-f19c-4e4b-800a-fdbf5cfccdf3 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:38.212159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6962-f19c-4e4b-800a-fdbf5cfccdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:38.212159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6962-f19c-4e4b-800a-fdbf5cfccdf3 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:38.213556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 607/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:23:38.218664 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f11f6-5034-44d9-8b2c-475be6be7d5d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:38.219099 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f11f6-5034-44d9-8b2c-475be6be7d5d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:38.219241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f11f6-5034-44d9-8b2c-475be6be7d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:38.239075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ec7f11f6-5034-44d9-8b2c-475be6be7d5d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:38.240249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ec7f11f6-5034-44d9-8b2c-475be6be7d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:38.248489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 599/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:38.532981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb6962-f19c-4e4b-800a-fdbf5cfccdf3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:38.534682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-90bb6962-f19c-4e4b-800a-fdbf5cfccdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:38.612629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb6962-f19c-4e4b-800a-fdbf5cfccdf3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:38.616112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5194c6d-6182-4b84-a0d2-169522eaad00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:38.616728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5194c6d-6182-4b84-a0d2-169522eaad00 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:38.617009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5194c6d-6182-4b84-a0d2-169522eaad00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:38.617277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5194c6d-6182-4b84-a0d2-169522eaad00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:38.625853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5194c6d-6182-4b84-a0d2-169522eaad00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:38.627269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5194c6d-6182-4b84-a0d2-169522eaad00 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:38.636530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 608/1207] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:40.787880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f2849-def5-4b7a-bb3d-b2c729f71d34 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:40.788435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f2849-def5-4b7a-bb3d-b2c729f71d34 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:40.788435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f2849-def5-4b7a-bb3d-b2c729f71d34 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:40.789680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 600/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:23:40.795420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb344727-da04-4493-a6aa-02867e1c07e5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:40.795868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb344727-da04-4493-a6aa-02867e1c07e5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:40.796010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb344727-da04-4493-a6aa-02867e1c07e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:40.816458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bb344727-da04-4493-a6aa-02867e1c07e5 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:40.817672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bb344727-da04-4493-a6aa-02867e1c07e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:40.825798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 609/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:41.118425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f2849-def5-4b7a-bb3d-b2c729f71d34 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:41.120194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4c9f2849-def5-4b7a-bb3d-b2c729f71d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:41.133481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f2849-def5-4b7a-bb3d-b2c729f71d34 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:41.137722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667f4ea-7713-400f-9f54-f06b824f902b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:41.138346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f4ea-7713-400f-9f54-f06b824f902b service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:41.138612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f4ea-7713-400f-9f54-f06b824f902b service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:41.138881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f4ea-7713-400f-9f54-f06b824f902b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:41.147669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667f4ea-7713-400f-9f54-f06b824f902b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:41.149098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667f4ea-7713-400f-9f54-f06b824f902b service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:41.158974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 601/1210] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:44.784701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c562eb-f874-4f0c-b8e4-3b7779c9ab01 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:44.785261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c562eb-f874-4f0c-b8e4-3b7779c9ab01 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:44.785261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c562eb-f874-4f0c-b8e4-3b7779c9ab01 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:44.786479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 610/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:23:44.792404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35615f5-bea0-42dc-a2b4-6fa4853671ba None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:44.792839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35615f5-bea0-42dc-a2b4-6fa4853671ba None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:44.792967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35615f5-bea0-42dc-a2b4-6fa4853671ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:44.811817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d35615f5-bea0-42dc-a2b4-6fa4853671ba None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:44.813104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d35615f5-bea0-42dc-a2b4-6fa4853671ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:44.820712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 602/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:45.123862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c562eb-f874-4f0c-b8e4-3b7779c9ab01 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:45.125547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85c562eb-f874-4f0c-b8e4-3b7779c9ab01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:45.171452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:23:45.202209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c562eb-f874-4f0c-b8e4-3b7779c9ab01 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:45.205957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827eb507-33e6-4a8d-b001-40f5c9e0982b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:45.206549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eb507-33e6-4a8d-b001-40f5c9e0982b service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:45.206827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eb507-33e6-4a8d-b001-40f5c9e0982b service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:45.207094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eb507-33e6-4a8d-b001-40f5c9e0982b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:45.216866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827eb507-33e6-4a8d-b001-40f5c9e0982b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:45.218285 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827eb507-33e6-4a8d-b001-40f5c9e0982b service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:45.227470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 611/1213] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:47.373152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad9d7f-3a9d-4982-9aca-16f048e985de None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:47.373714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad9d7f-3a9d-4982-9aca-16f048e985de None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:47.373714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad9d7f-3a9d-4982-9aca-16f048e985de None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:47.374954 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 603/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:23:47.380233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4e27-6837-45d3-8355-d0636f8e95d4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:47.380702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4e27-6837-45d3-8355-d0636f8e95d4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:47.380929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4e27-6837-45d3-8355-d0636f8e95d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:47.400591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b57c4e27-6837-45d3-8355-d0636f8e95d4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:47.402497 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b57c4e27-6837-45d3-8355-d0636f8e95d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:47.410294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 612/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:47.713159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad9d7f-3a9d-4982-9aca-16f048e985de None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:47.714836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-55ad9d7f-3a9d-4982-9aca-16f048e985de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:47.727670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad9d7f-3a9d-4982-9aca-16f048e985de None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:47.731161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51e3e6-5b75-4c14-a071-2c37dd10ac78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:47.731747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51e3e6-5b75-4c14-a071-2c37dd10ac78 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:47.732023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51e3e6-5b75-4c14-a071-2c37dd10ac78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:47.732319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51e3e6-5b75-4c14-a071-2c37dd10ac78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:47.740949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51e3e6-5b75-4c14-a071-2c37dd10ac78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:47.742352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51e3e6-5b75-4c14-a071-2c37dd10ac78 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:47.752143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 604/1216] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:57.395265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=18 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:23:57.748021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=82 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:11.511412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5b7c-2936-4495-92ca-a1341ecf3e3e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.512714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5b7c-2936-4495-92ca-a1341ecf3e3e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.513256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed5b7c-2936-4495-92ca-a1341ecf3e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.540026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9eed5b7c-2936-4495-92ca-a1341ecf3e3e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.541718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9eed5b7c-2936-4495-92ca-a1341ecf3e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.559153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 613/1217] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.568837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73883c4a-30dc-48ba-8027-ba42d79d79fd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.569451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73883c4a-30dc-48ba-8027-ba42d79d79fd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.569451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73883c4a-30dc-48ba-8027-ba42d79d79fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.571626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337650-8db8-4dd1-a437-8ae0689e0138 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.572150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337650-8db8-4dd1-a437-8ae0689e0138 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.572675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337650-8db8-4dd1-a437-8ae0689e0138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.601488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fe337650-8db8-4dd1-a437-8ae0689e0138 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.603101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe337650-8db8-4dd1-a437-8ae0689e0138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.608290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-73883c4a-30dc-48ba-8027-ba42d79d79fd None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.608290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-73883c4a-30dc-48ba-8027-ba42d79d79fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.619337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 614/1218] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:25:11.621014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eed07-4846-4a7a-b8de-55213b15b64f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.621527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eed07-4846-4a7a-b8de-55213b15b64f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.622015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eed07-4846-4a7a-b8de-55213b15b64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.624232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 605/1219] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.628301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a6f1-7063-4040-8d74-b680cfce8d0f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.628301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a6f1-7063-4040-8d74-b680cfce8d0f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.628301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0a6f1-7063-4040-8d74-b680cfce8d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.653347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-109eed07-4846-4a7a-b8de-55213b15b64f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.654951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-109eed07-4846-4a7a-b8de-55213b15b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.669262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bdd0a6f1-7063-4040-8d74-b680cfce8d0f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.669738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 615/1220] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.670770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109eed07-4846-4a7a-b8de-55213b15b64f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:11.670883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bdd0a6f1-7063-4040-8d74-b680cfce8d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.686667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109eed07-4846-4a7a-b8de-55213b15b64f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:11.693989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 606/1221] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.695260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b98f0-a9fc-456a-ba35-f350e52d3714 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.695545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b98f0-a9fc-456a-ba35-f350e52d3714 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.695820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b98f0-a9fc-456a-ba35-f350e52d3714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.697948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad38d2-e680-4ed9-815b-0d4c7d4021c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:11.698807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad38d2-e680-4ed9-815b-0d4c7d4021c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.699386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad38d2-e680-4ed9-815b-0d4c7d4021c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.699885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad38d2-e680-4ed9-815b-0d4c7d4021c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.700778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad38d2-e680-4ed9-815b-0d4c7d4021c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:11.701287 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad38d2-e680-4ed9-815b-0d4c7d4021c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:11.703316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad38d2-e680-4ed9-815b-0d4c7d4021c7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:11.726118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 616/1222] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:11.732296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f9b9c-2b7d-44c6-ad7a-2468e7fd5015 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.732883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f9b9c-2b7d-44c6-ad7a-2468e7fd5015 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.733024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f9b9c-2b7d-44c6-ad7a-2468e7fd5015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.739949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cd3b98f0-a9fc-456a-ba35-f350e52d3714 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.741202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cd3b98f0-a9fc-456a-ba35-f350e52d3714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.762040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 607/1223] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.762237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-363f9b9c-2b7d-44c6-ad7a-2468e7fd5015 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.763222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a96b-d787-42ec-b0af-d17116038000 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.763334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-363f9b9c-2b7d-44c6-ad7a-2468e7fd5015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.763556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a96b-d787-42ec-b0af-d17116038000 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.763776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8a96b-d787-42ec-b0af-d17116038000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.781157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 617/1224] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.781969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f9b9c-2b7d-44c6-ad7a-2468e7fd5015 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:11.794854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f9b9c-2b7d-44c6-ad7a-2468e7fd5015 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:11.798693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-14a8a96b-d787-42ec-b0af-d17116038000 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.799529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8612d5-d2c9-48b5-8d31-1dd6489f03d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:11.799895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-14a8a96b-d787-42ec-b0af-d17116038000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.800123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8612d5-d2c9-48b5-8d31-1dd6489f03d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.800431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8612d5-d2c9-48b5-8d31-1dd6489f03d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.800762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8612d5-d2c9-48b5-8d31-1dd6489f03d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.801323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8612d5-d2c9-48b5-8d31-1dd6489f03d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:11.802327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-5a8612d5-d2c9-48b5-8d31-1dd6489f03d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:25:11.802327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:25:11.802327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:25:11.803620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8612d5-d2c9-48b5-8d31-1dd6489f03d0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:11.810976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 608/1225] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.812196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec756c-d063-45ea-af29-ff6894689e27 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.812505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec756c-d063-45ea-af29-ff6894689e27 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.812837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec756c-d063-45ea-af29-ff6894689e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.832847 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9eec756c-d063-45ea-af29-ff6894689e27 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.834067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9eec756c-d063-45ea-af29-ff6894689e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.842474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 609/1226] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.843231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec756c-d063-45ea-af29-ff6894689e27 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:11.853100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec756c-d063-45ea-af29-ff6894689e27 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:11.857752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad215f8-813f-4286-9d53-a4832401ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:11.858329 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad215f8-813f-4286-9d53-a4832401ca6d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.858606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad215f8-813f-4286-9d53-a4832401ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.858878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad215f8-813f-4286-9d53-a4832401ca6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.859587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad215f8-813f-4286-9d53-a4832401ca6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:11.859840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad215f8-813f-4286-9d53-a4832401ca6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:11.860868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-3ad215f8-813f-4286-9d53-a4832401ca6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:25:11.860868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Oct 24 13:25:11.860868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: Oct 24 13:25:11.861741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad215f8-813f-4286-9d53-a4832401ca6d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:11.873169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 610/1227] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:11.874451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe62bb1-64d0-4f8b-97cd-aaf7fed2307a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.874726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe62bb1-64d0-4f8b-97cd-aaf7fed2307a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.875003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe62bb1-64d0-4f8b-97cd-aaf7fed2307a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.894388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dbe62bb1-64d0-4f8b-97cd-aaf7fed2307a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.895570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dbe62bb1-64d0-4f8b-97cd-aaf7fed2307a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.904118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 611/1228] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.912875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7c6f-440f-4117-8a64-d5ac2e0039a5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.912875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7c6f-440f-4117-8a64-d5ac2e0039a5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.912875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7c6f-440f-4117-8a64-d5ac2e0039a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.924773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-501d7c6f-440f-4117-8a64-d5ac2e0039a5 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.927153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-501d7c6f-440f-4117-8a64-d5ac2e0039a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:11.933772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 612/1229] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:11.934582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7c6f-440f-4117-8a64-d5ac2e0039a5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:11.943746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7c6f-440f-4117-8a64-d5ac2e0039a5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:11.949081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:11.949665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.949947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.950221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.950859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:11.951168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:11.952810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:11.958363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 618/1230] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/projects => generated 389 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:11.958363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e937-eb80-415b-91a0-50917cacb953 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:11.958363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e937-eb80-415b-91a0-50917cacb953 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:11.958363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821e937-eb80-415b-91a0-50917cacb953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:11.969248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 613/1231] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:11.969927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:11.979481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b303a8-41a8-463f-8293-a02cf59165d9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:11.983074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d821e937-eb80-415b-91a0-50917cacb953 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:11.984567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d821e937-eb80-415b-91a0-50917cacb953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:12.035014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.035540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.035822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.036090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.036758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:12.037066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.038640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.042329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 619/1232] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:12.043648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09ee01-248e-43a5-b307-37ec4d9fa264 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.043936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09ee01-248e-43a5-b307-37ec4d9fa264 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.044214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09ee01-248e-43a5-b307-37ec4d9fa264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.055534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 614/1233] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:12.055534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.065284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3b09ee01-248e-43a5-b307-37ec4d9fa264 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:12.066080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3b09ee01-248e-43a5-b307-37ec4d9fa264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:12.072355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93386d9c-50dd-43b2-a659-a8958a4f5704 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.074961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 620/1234] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:12.076183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08baf6-a016-46bd-b923-bc8c59782f52 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.076629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08baf6-a016-46bd-b923-bc8c59782f52 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.076855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08baf6-a016-46bd-b923-bc8c59782f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.084988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6e242-1824-467a-983b-935df625b120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.087898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6e242-1824-467a-983b-935df625b120 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.087898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6e242-1824-467a-983b-935df625b120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.087898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6e242-1824-467a-983b-935df625b120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.087898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6e242-1824-467a-983b-935df625b120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.089567 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6e242-1824-467a-983b-935df625b120 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.097502 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4a08baf6-a016-46bd-b923-bc8c59782f52 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:12.098693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4a08baf6-a016-46bd-b923-bc8c59782f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:12.106817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 621/1235] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:12.109556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acfa32-3dfb-4e2a-9bf3-cb3df8e7a1ba None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.109840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acfa32-3dfb-4e2a-9bf3-cb3df8e7a1ba None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.110140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acfa32-3dfb-4e2a-9bf3-cb3df8e7a1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.129204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b1acfa32-3dfb-4e2a-9bf3-cb3df8e7a1ba None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:12.130368 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b1acfa32-3dfb-4e2a-9bf3-cb3df8e7a1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:12.138543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 622/1236] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:12.143893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acfa32-3dfb-4e2a-9bf3-cb3df8e7a1ba None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.148235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acfa32-3dfb-4e2a-9bf3-cb3df8e7a1ba None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.153394 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc620d76-c025-4d77-95c7-b9245f98f6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.153957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc620d76-c025-4d77-95c7-b9245f98f6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.154234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc620d76-c025-4d77-95c7-b9245f98f6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.154505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc620d76-c025-4d77-95c7-b9245f98f6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.155107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc620d76-c025-4d77-95c7-b9245f98f6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.157097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc620d76-c025-4d77-95c7-b9245f98f6f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.172311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 615/1237] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.173973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6e242-1824-467a-983b-935df625b120 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.233017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6e242-1824-467a-983b-935df625b120 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.237624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eebef9-54de-4d8f-8bbd-936bff7583de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.238195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eebef9-54de-4d8f-8bbd-936bff7583de admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.238473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eebef9-54de-4d8f-8bbd-936bff7583de admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.238746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eebef9-54de-4d8f-8bbd-936bff7583de admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.239359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eebef9-54de-4d8f-8bbd-936bff7583de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.241283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eebef9-54de-4d8f-8bbd-936bff7583de admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.246223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:12.251150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 623/1238] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.252508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c479a7-5c03-44b7-8339-8ec376210416 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.258403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c479a7-5c03-44b7-8339-8ec376210416 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.258403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c479a7-5c03-44b7-8339-8ec376210416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.272620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-03c479a7-5c03-44b7-8339-8ec376210416 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:12.273883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-03c479a7-5c03-44b7-8339-8ec376210416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:12.281581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 624/1239] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:12.286810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c479a7-5c03-44b7-8339-8ec376210416 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.291856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c479a7-5c03-44b7-8339-8ec376210416 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.296420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d2a87-b3f3-4422-8466-128f1dd06ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.296995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d2a87-b3f3-4422-8466-128f1dd06ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.297267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d2a87-b3f3-4422-8466-128f1dd06ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.297536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d2a87-b3f3-4422-8466-128f1dd06ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.298146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d2a87-b3f3-4422-8466-128f1dd06ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.300026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d2a87-b3f3-4422-8466-128f1dd06ab5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.342759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 616/1240] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/projects => generated 391 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.343595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eebef9-54de-4d8f-8bbd-936bff7583de admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.403900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eebef9-54de-4d8f-8bbd-936bff7583de admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.408618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.409277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.409558 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.409833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.410473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:12.411022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.412428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.420670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 625/1241] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.420670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d2a87-b3f3-4422-8466-128f1dd06ab5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.432669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 617/1242] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:12.437264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.447932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:12.448110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b35134-9967-4bcc-8ebd-2c456ab8c43e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.482018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d2a87-b3f3-4422-8466-128f1dd06ab5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.486679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.487401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.487601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.487827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.488557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:12.488971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.490437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.509806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 626/1243] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:12.509806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.510125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.510125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.510125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.510125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.510486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:12.510486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.510486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.515751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03f7d6-56dd-4f44-b34e-b9053c778e17 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.520374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ec0f1-da06-43b8-8654-e445d47cb26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.520564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 618/1244] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:12.520954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec0f1-da06-43b8-8654-e445d47cb26d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.521241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec0f1-da06-43b8-8654-e445d47cb26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.521503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec0f1-da06-43b8-8654-e445d47cb26d admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.521625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.522120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ec0f1-da06-43b8-8654-e445d47cb26d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.524038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ec0f1-da06-43b8-8654-e445d47cb26d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.533501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818af02-6364-4d4b-8ed9-5d2d29dd42be admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.538172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c3b5c-4341-441f-8571-c4aed817661e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.538698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3b5c-4341-441f-8571-c4aed817661e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.538977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3b5c-4341-441f-8571-c4aed817661e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.539252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3b5c-4341-441f-8571-c4aed817661e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.539879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c3b5c-4341-441f-8571-c4aed817661e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.541479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c3b5c-4341-441f-8571-c4aed817661e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.551865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 619/1245] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:12.552732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c3b5c-4341-441f-8571-c4aed817661e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.562405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c3b5c-4341-441f-8571-c4aed817661e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.566965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.567513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.567794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.568063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.568761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:12.569062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.570588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.582696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 620/1246] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:12.583429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.592488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970182fc-68a7-4381-8045-63683d1f79be admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.597040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.597585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.597860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.598130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.598729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.599698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:25:12.599698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Oct 24 13:25:12.599698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: Oct 24 13:25:12.600918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.607771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 627/1247] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.607771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ec0f1-da06-43b8-8654-e445d47cb26d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.615385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ec0f1-da06-43b8-8654-e445d47cb26d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.620684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7f4b4-9d1c-40c3-8c72-e0908f3dbf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.621262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7f4b4-9d1c-40c3-8c72-e0908f3dbf20 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.621537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7f4b4-9d1c-40c3-8c72-e0908f3dbf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.621803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7f4b4-9d1c-40c3-8c72-e0908f3dbf20 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.622411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7f4b4-9d1c-40c3-8c72-e0908f3dbf20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.624312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7f4b4-9d1c-40c3-8c72-e0908f3dbf20 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.661272 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 621/1248] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/users => generated 296 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.662072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.671935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185d720-c59e-4413-8b72-f881c381339f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.676531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1166de-67e1-458f-8faa-958bf54ff0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.681070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1166de-67e1-458f-8faa-958bf54ff0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.681070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1166de-67e1-458f-8faa-958bf54ff0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.681070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1166de-67e1-458f-8faa-958bf54ff0e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.681070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1166de-67e1-458f-8faa-958bf54ff0e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.681070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1166de-67e1-458f-8faa-958bf54ff0e2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.702344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 628/1249] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.703213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7f4b4-9d1c-40c3-8c72-e0908f3dbf20 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.712846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7f4b4-9d1c-40c3-8c72-e0908f3dbf20 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.717332 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8293d-300f-475e-bb79-5cab000031bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.717890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8293d-300f-475e-bb79-5cab000031bf admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.718164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8293d-300f-475e-bb79-5cab000031bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.718431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8293d-300f-475e-bb79-5cab000031bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.719028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8293d-300f-475e-bb79-5cab000031bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.720938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8293d-300f-475e-bb79-5cab000031bf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.753865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 622/1250] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/projects => generated 367 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.754675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1166de-67e1-458f-8faa-958bf54ff0e2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.764164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:12.803194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 629/1251] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.803194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8293d-300f-475e-bb79-5cab000031bf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.814874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1166de-67e1-458f-8faa-958bf54ff0e2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.864505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8293d-300f-475e-bb79-5cab000031bf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.869586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b76c7-ac62-49d7-bba6-d2ee0dc3dfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.870178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b76c7-ac62-49d7-bba6-d2ee0dc3dfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.870454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b76c7-ac62-49d7-bba6-d2ee0dc3dfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.870972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b76c7-ac62-49d7-bba6-d2ee0dc3dfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.871555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b76c7-ac62-49d7-bba6-d2ee0dc3dfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.871996 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390f7c-e809-4ac4-887c-00b70f744761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.872631 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390f7c-e809-4ac4-887c-00b70f744761 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.872907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390f7c-e809-4ac4-887c-00b70f744761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.873174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390f7c-e809-4ac4-887c-00b70f744761 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.873312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b76c7-ac62-49d7-bba6-d2ee0dc3dfd3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.873794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1390f7c-e809-4ac4-887c-00b70f744761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.875721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1390f7c-e809-4ac4-887c-00b70f744761 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.882950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 630/1252] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:12.883827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b76c7-ac62-49d7-bba6-d2ee0dc3dfd3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:12.893195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b76c7-ac62-49d7-bba6-d2ee0dc3dfd3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:12.897632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba5519-4376-45b3-a286-467d5f897e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:12.902878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5519-4376-45b3-a286-467d5f897e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:12.902878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5519-4376-45b3-a286-467d5f897e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:12.902878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5519-4376-45b3-a286-467d5f897e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:12.902878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba5519-4376-45b3-a286-467d5f897e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:12.902878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba5519-4376-45b3-a286-467d5f897e35 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:12.996550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 623/1253] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:12.996550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390f7c-e809-4ac4-887c-00b70f744761 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.010566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 631/1254] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:12 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:13.014448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba5519-4376-45b3-a286-467d5f897e35 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.050508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:13.060401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390f7c-e809-4ac4-887c-00b70f744761 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.071466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba5519-4376-45b3-a286-467d5f897e35 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.076478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc7edb-6556-49ae-b1e7-cf6a955e7e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.077596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc7edb-6556-49ae-b1e7-cf6a955e7e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.077596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc7edb-6556-49ae-b1e7-cf6a955e7e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.077596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc7edb-6556-49ae-b1e7-cf6a955e7e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.078203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc7edb-6556-49ae-b1e7-cf6a955e7e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.079786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc7edb-6556-49ae-b1e7-cf6a955e7e5a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.096493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 632/1255] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.097568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc7edb-6556-49ae-b1e7-cf6a955e7e5a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.107904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc7edb-6556-49ae-b1e7-cf6a955e7e5a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.109833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:13.112530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3be062-8529-4da4-90ec-4fddf8399ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.113109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3be062-8529-4da4-90ec-4fddf8399ead admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.113388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3be062-8529-4da4-90ec-4fddf8399ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.113659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3be062-8529-4da4-90ec-4fddf8399ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.114283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3be062-8529-4da4-90ec-4fddf8399ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.115871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3be062-8529-4da4-90ec-4fddf8399ead admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.121303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc443b-dc90-4dae-bb66-d16a3545b152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.121895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc443b-dc90-4dae-bb66-d16a3545b152 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.122214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc443b-dc90-4dae-bb66-d16a3545b152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.122482 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc443b-dc90-4dae-bb66-d16a3545b152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.123110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc443b-dc90-4dae-bb66-d16a3545b152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.124734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc443b-dc90-4dae-bb66-d16a3545b152 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.125795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 633/1256] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.126559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3be062-8529-4da4-90ec-4fddf8399ead admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.135200 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 624/1257] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.135844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc443b-dc90-4dae-bb66-d16a3545b152 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.140169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3be062-8529-4da4-90ec-4fddf8399ead admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.144850 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774c6c-dec5-48de-8ca3-87e02d8d9be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.145248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc443b-dc90-4dae-bb66-d16a3545b152 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.145431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774c6c-dec5-48de-8ca3-87e02d8d9be7 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.145708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774c6c-dec5-48de-8ca3-87e02d8d9be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.146009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a774c6c-dec5-48de-8ca3-87e02d8d9be7 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.146648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774c6c-dec5-48de-8ca3-87e02d8d9be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.148604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a774c6c-dec5-48de-8ca3-87e02d8d9be7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.151137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2d9a7-97e9-4482-9f01-3af4afe27ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.152150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2d9a7-97e9-4482-9f01-3af4afe27ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.152563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2d9a7-97e9-4482-9f01-3af4afe27ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.153618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2d9a7-97e9-4482-9f01-3af4afe27ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.154259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2d9a7-97e9-4482-9f01-3af4afe27ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.157393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2d9a7-97e9-4482-9f01-3af4afe27ba7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.238686 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 634/1258] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:13 2022] POST /identity/v3/users => generated 284 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:13.242675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774c6c-dec5-48de-8ca3-87e02d8d9be7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.243594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 625/1259] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:13 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:13.244294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2d9a7-97e9-4482-9f01-3af4afe27ba7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.252687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a774c6c-dec5-48de-8ca3-87e02d8d9be7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.255248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2d9a7-97e9-4482-9f01-3af4afe27ba7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.257241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70825a6d-eb47-4880-a89b-1458383742b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.257918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70825a6d-eb47-4880-a89b-1458383742b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.258193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70825a6d-eb47-4880-a89b-1458383742b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.258462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70825a6d-eb47-4880-a89b-1458383742b4 admin admin] PATH_INFO: `/v3/projects/7fa2196e343c45b5b741c39417ba9ef8/users/7c096064e6ec443a9e8755bc2e68dbb7/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.259810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33025e64-c35b-4dab-822c-3771dac8644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.262255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33025e64-c35b-4dab-822c-3771dac8644c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.262255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33025e64-c35b-4dab-822c-3771dac8644c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.262255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33025e64-c35b-4dab-822c-3771dac8644c admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.262255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33025e64-c35b-4dab-822c-3771dac8644c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.265103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33025e64-c35b-4dab-822c-3771dac8644c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.276187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70825a6d-eb47-4880-a89b-1458383742b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa2196e343c45b5b741c39417ba9ef8, user_id=7c096064e6ec443a9e8755bc2e68dbb7, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.278884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70825a6d-eb47-4880-a89b-1458383742b4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.342248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 626/1260] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:13 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:13.342976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33025e64-c35b-4dab-822c-3771dac8644c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.352333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33025e64-c35b-4dab-822c-3771dac8644c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.356884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f333f3e-34f4-4bfd-ab3b-66be61a28695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.357455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f333f3e-34f4-4bfd-ab3b-66be61a28695 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.357734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f333f3e-34f4-4bfd-ab3b-66be61a28695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.358003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f333f3e-34f4-4bfd-ab3b-66be61a28695 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.358614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f333f3e-34f4-4bfd-ab3b-66be61a28695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.360525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f333f3e-34f4-4bfd-ab3b-66be61a28695 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.363902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 635/1261] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:13 2022] PUT /identity/v3/projects/7fa2196e343c45b5b741c39417ba9ef8/users/7c096064e6ec443a9e8755bc2e68dbb7/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:13.364787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70825a6d-eb47-4880-a89b-1458383742b4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.427297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70825a6d-eb47-4880-a89b-1458383742b4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.431907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03b2d7-f1bc-4748-ad9a-f215413212aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.432541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03b2d7-f1bc-4748-ad9a-f215413212aa admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.432816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03b2d7-f1bc-4748-ad9a-f215413212aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.433087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03b2d7-f1bc-4748-ad9a-f215413212aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.433742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03b2d7-f1bc-4748-ad9a-f215413212aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.435348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03b2d7-f1bc-4748-ad9a-f215413212aa admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.445040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 636/1262] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.445786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03b2d7-f1bc-4748-ad9a-f215413212aa admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.450648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 627/1263] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:13 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:13.454089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f333f3e-34f4-4bfd-ab3b-66be61a28695 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.456080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03b2d7-f1bc-4748-ad9a-f215413212aa admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.461218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18889-23be-449e-a212-86e2133552fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.462312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f333f3e-34f4-4bfd-ab3b-66be61a28695 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.462445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18889-23be-449e-a212-86e2133552fe admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.462689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18889-23be-449e-a212-86e2133552fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.463077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18889-23be-449e-a212-86e2133552fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.463738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18889-23be-449e-a212-86e2133552fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.465348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18889-23be-449e-a212-86e2133552fe admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.467047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39219e83-c496-4842-b1ad-0dedefb3040e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.467622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39219e83-c496-4842-b1ad-0dedefb3040e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.467898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39219e83-c496-4842-b1ad-0dedefb3040e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.468169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39219e83-c496-4842-b1ad-0dedefb3040e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.468851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39219e83-c496-4842-b1ad-0dedefb3040e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.470390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39219e83-c496-4842-b1ad-0dedefb3040e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.475740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 637/1264] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.477197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18889-23be-449e-a212-86e2133552fe admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.482859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 628/1265] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.482859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39219e83-c496-4842-b1ad-0dedefb3040e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.490364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18889-23be-449e-a212-86e2133552fe admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.492174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39219e83-c496-4842-b1ad-0dedefb3040e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.494955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400c1d-ac45-4410-a7b9-2401863a11a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.495530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400c1d-ac45-4410-a7b9-2401863a11a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.495798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400c1d-ac45-4410-a7b9-2401863a11a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.496065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400c1d-ac45-4410-a7b9-2401863a11a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.496715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400c1d-ac45-4410-a7b9-2401863a11a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.496823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dbf27-d7b4-4dc4-ba20-023a57249536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.497368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dbf27-d7b4-4dc4-ba20-023a57249536 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.497671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dbf27-d7b4-4dc4-ba20-023a57249536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.497941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dbf27-d7b4-4dc4-ba20-023a57249536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.498268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400c1d-ac45-4410-a7b9-2401863a11a0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.498557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dbf27-d7b4-4dc4-ba20-023a57249536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.500102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474dbf27-d7b4-4dc4-ba20-023a57249536 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.513658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 638/1266] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.513831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 629/1267] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.513936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400c1d-ac45-4410-a7b9-2401863a11a0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.514024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89798e86-0232-48ab-80c0-e6148265c71a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.514024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89798e86-0232-48ab-80c0-e6148265c71a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.514024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89798e86-0232-48ab-80c0-e6148265c71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.522578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400c1d-ac45-4410-a7b9-2401863a11a0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.527828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844f43d-cd26-4b9b-ab71-d2e74455ea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.528442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f43d-cd26-4b9b-ab71-d2e74455ea80 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.528723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f43d-cd26-4b9b-ab71-d2e74455ea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.528992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844f43d-cd26-4b9b-ab71-d2e74455ea80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.529624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844f43d-cd26-4b9b-ab71-d2e74455ea80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.531190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844f43d-cd26-4b9b-ab71-d2e74455ea80 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.533462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-89798e86-0232-48ab-80c0-e6148265c71a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:13.534681 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-89798e86-0232-48ab-80c0-e6148265c71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:13.540995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 639/1268] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.541751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844f43d-cd26-4b9b-ab71-d2e74455ea80 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.545904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 630/1269] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:13.546677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89798e86-0232-48ab-80c0-e6148265c71a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.553978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844f43d-cd26-4b9b-ab71-d2e74455ea80 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.556183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89798e86-0232-48ab-80c0-e6148265c71a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.558771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ea65b-e839-430f-a618-eb0fe3694258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.559416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea65b-e839-430f-a618-eb0fe3694258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.559696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea65b-e839-430f-a618-eb0fe3694258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.559972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ea65b-e839-430f-a618-eb0fe3694258 admin admin] PATH_INFO: `/v3/projects/986792b802264c748e4614c4d2a17680/users/633c6c68e15f4be9a7289d15cac9de43/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.561217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747442d-e13f-4b54-9bc4-14661fc2879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.561834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747442d-e13f-4b54-9bc4-14661fc2879b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.562110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747442d-e13f-4b54-9bc4-14661fc2879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.562379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747442d-e13f-4b54-9bc4-14661fc2879b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.563003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747442d-e13f-4b54-9bc4-14661fc2879b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.564624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747442d-e13f-4b54-9bc4-14661fc2879b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.575296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ea65b-e839-430f-a618-eb0fe3694258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=986792b802264c748e4614c4d2a17680, user_id=633c6c68e15f4be9a7289d15cac9de43, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.579382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 631/1270] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.579382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747442d-e13f-4b54-9bc4-14661fc2879b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.579908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ea65b-e839-430f-a618-eb0fe3694258 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.587570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747442d-e13f-4b54-9bc4-14661fc2879b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.592130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e743ad-2ba3-48ee-992d-67b5724127b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.592835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e743ad-2ba3-48ee-992d-67b5724127b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.593057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e743ad-2ba3-48ee-992d-67b5724127b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.593334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e743ad-2ba3-48ee-992d-67b5724127b7 admin admin] PATH_INFO: `/v3/projects/60e0c5c137ef4274a8c6ee990b4d8296/users/706e3bfc697d4073b8f9cf6c98f98147/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.607354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e743ad-2ba3-48ee-992d-67b5724127b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e0c5c137ef4274a8c6ee990b4d8296, user_id=706e3bfc697d4073b8f9cf6c98f98147, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.611040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e743ad-2ba3-48ee-992d-67b5724127b7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.655998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 640/1271] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:13 2022] PUT /identity/v3/projects/986792b802264c748e4614c4d2a17680/users/633c6c68e15f4be9a7289d15cac9de43/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:13.656952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ea65b-e839-430f-a618-eb0fe3694258 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.675598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:13.687875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 632/1272] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:13 2022] PUT /identity/v3/projects/60e0c5c137ef4274a8c6ee990b4d8296/users/706e3bfc697d4073b8f9cf6c98f98147/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:13.687875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e743ad-2ba3-48ee-992d-67b5724127b7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.720998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ea65b-e839-430f-a618-eb0fe3694258 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.746675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e743ad-2ba3-48ee-992d-67b5724127b7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.751698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5984525-1132-4bec-8408-e7e0041fa6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.752339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5984525-1132-4bec-8408-e7e0041fa6be admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.752620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5984525-1132-4bec-8408-e7e0041fa6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.752891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5984525-1132-4bec-8408-e7e0041fa6be admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.753669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5984525-1132-4bec-8408-e7e0041fa6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.755358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5984525-1132-4bec-8408-e7e0041fa6be admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.766109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 633/1273] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.766922 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5984525-1132-4bec-8408-e7e0041fa6be admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.776480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5984525-1132-4bec-8408-e7e0041fa6be admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.778762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e932eb-12f7-49cf-bd6f-c53c57fa30be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.779417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e932eb-12f7-49cf-bd6f-c53c57fa30be admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.779740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e932eb-12f7-49cf-bd6f-c53c57fa30be admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.780326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e932eb-12f7-49cf-bd6f-c53c57fa30be admin admin] PATH_INFO: `/v3/projects/f14cde250a534b1dab61966cb9970c2e/users/9939003319344ccb8cde9e9e8bab9649/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.781124 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522663c-1986-45c8-a86c-9ac7ff1373b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.781696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522663c-1986-45c8-a86c-9ac7ff1373b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.781972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522663c-1986-45c8-a86c-9ac7ff1373b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.782244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522663c-1986-45c8-a86c-9ac7ff1373b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.782862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522663c-1986-45c8-a86c-9ac7ff1373b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.784456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522663c-1986-45c8-a86c-9ac7ff1373b3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.794550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 634/1274] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.795358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522663c-1986-45c8-a86c-9ac7ff1373b3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.797648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e932eb-12f7-49cf-bd6f-c53c57fa30be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14cde250a534b1dab61966cb9970c2e, user_id=9939003319344ccb8cde9e9e8bab9649, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.800349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e932eb-12f7-49cf-bd6f-c53c57fa30be admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.804789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522663c-1986-45c8-a86c-9ac7ff1373b3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.809787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e9241-fabe-4286-aa5b-b9a999d453dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.814024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e9241-fabe-4286-aa5b-b9a999d453dc admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.814024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e9241-fabe-4286-aa5b-b9a999d453dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.814024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e9241-fabe-4286-aa5b-b9a999d453dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.814024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e9241-fabe-4286-aa5b-b9a999d453dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.814024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e9241-fabe-4286-aa5b-b9a999d453dc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.823887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 635/1275] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.824821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e9241-fabe-4286-aa5b-b9a999d453dc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.828841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:13.877058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e9241-fabe-4286-aa5b-b9a999d453dc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.880538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e4818-6968-4586-8551-cd4d80169c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.881177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e4818-6968-4586-8551-cd4d80169c3a service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.881458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e4818-6968-4586-8551-cd4d80169c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.881738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e4818-6968-4586-8551-cd4d80169c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.885456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 641/1276] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:13 2022] PUT /identity/v3/projects/f14cde250a534b1dab61966cb9970c2e/users/9939003319344ccb8cde9e9e8bab9649/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:13.886295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e932eb-12f7-49cf-bd6f-c53c57fa30be admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:13.904016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e4818-6968-4586-8551-cd4d80169c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.905588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e4818-6968-4586-8551-cd4d80169c3a service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.917968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 636/1277] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:13.919305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7d10-938a-4993-9750-03c8c9016af4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.919590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7d10-938a-4993-9750-03c8c9016af4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.919865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe7d10-938a-4993-9750-03c8c9016af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.941424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-60fe7d10-938a-4993-9750-03c8c9016af4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:13.942672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-60fe7d10-938a-4993-9750-03c8c9016af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:13.965155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e932eb-12f7-49cf-bd6f-c53c57fa30be admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:13.969822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1160c-1be4-4b5c-942d-e36a2ba2d454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:13.970417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1160c-1be4-4b5c-942d-e36a2ba2d454 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.970690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1160c-1be4-4b5c-942d-e36a2ba2d454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.970963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1160c-1be4-4b5c-942d-e36a2ba2d454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:13.971592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1160c-1be4-4b5c-942d-e36a2ba2d454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:13.973196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1160c-1be4-4b5c-942d-e36a2ba2d454 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:13.983316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 642/1278] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:13.986545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ba49-04e2-4aa2-a8b1-afc8402a400a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:13.986545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ba49-04e2-4aa2-a8b1-afc8402a400a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:13.986741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ba49-04e2-4aa2-a8b1-afc8402a400a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.004138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 637/1279] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:14.007721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe7d10-938a-4993-9750-03c8c9016af4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.018689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8c54ba49-04e2-4aa2-a8b1-afc8402a400a None None] MFA Rules not processed for user `706e3bfc697d4073b8f9cf6c98f98147`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:14.020188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8c54ba49-04e2-4aa2-a8b1-afc8402a400a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:14.022650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe7d10-938a-4993-9750-03c8c9016af4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.027499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60bf14-dd20-4231-bc2b-91fae1577108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.028154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60bf14-dd20-4231-bc2b-91fae1577108 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.028842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60bf14-dd20-4231-bc2b-91fae1577108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.028842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60bf14-dd20-4231-bc2b-91fae1577108 admin admin] PATH_INFO: `/v3/projects/e3faf7ad1fea4ab8b08c98a10491ec11/users/e0dd5e32e4a642c8ac9e44310e0a6a5b/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.044487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60bf14-dd20-4231-bc2b-91fae1577108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3faf7ad1fea4ab8b08c98a10491ec11, user_id=e0dd5e32e4a642c8ac9e44310e0a6a5b, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.047798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60bf14-dd20-4231-bc2b-91fae1577108 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.059038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:14.092434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 643/1280] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:14.093268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ba49-04e2-4aa2-a8b1-afc8402a400a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.104990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ba49-04e2-4aa2-a8b1-afc8402a400a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.108856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ebab7-9178-4e08-9f84-bfd81c981d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.109495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ebab7-9178-4e08-9f84-bfd81c981d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.109774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ebab7-9178-4e08-9f84-bfd81c981d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.110048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ebab7-9178-4e08-9f84-bfd81c981d86 admin admin] PATH_INFO: `/v3/projects/a2579a5bb0884e9c8a4d16d3e5dc6fcf/users/68fec4ef4a8549e5a4c5a79b296551a5/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.124959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ebab7-9178-4e08-9f84-bfd81c981d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2579a5bb0884e9c8a4d16d3e5dc6fcf, user_id=68fec4ef4a8549e5a4c5a79b296551a5, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.127692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ebab7-9178-4e08-9f84-bfd81c981d86 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.130253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 638/1281] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:14 2022] PUT /identity/v3/projects/e3faf7ad1fea4ab8b08c98a10491ec11/users/e0dd5e32e4a642c8ac9e44310e0a6a5b/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:14.131091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60bf14-dd20-4231-bc2b-91fae1577108 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.191789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60bf14-dd20-4231-bc2b-91fae1577108 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.196508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddbe74-9b2e-4707-9773-378bbd3688ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.204376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddbe74-9b2e-4707-9773-378bbd3688ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.204376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddbe74-9b2e-4707-9773-378bbd3688ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.204376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddbe74-9b2e-4707-9773-378bbd3688ba admin admin] PATH_INFO: `/v3/projects/9b9bec5e97434a5a89f496d2349549df/users/601cc4a65cee4a86a421da37f6a980cb/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.211958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddbe74-9b2e-4707-9773-378bbd3688ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9bec5e97434a5a89f496d2349549df, user_id=601cc4a65cee4a86a421da37f6a980cb, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.214653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddbe74-9b2e-4707-9773-378bbd3688ba admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.221435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 644/1282] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:14 2022] PUT /identity/v3/projects/a2579a5bb0884e9c8a4d16d3e5dc6fcf/users/68fec4ef4a8549e5a4c5a79b296551a5/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:14.223000 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec3bad-5819-4246-a8cb-ca147f08cb80 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.223283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec3bad-5819-4246-a8cb-ca147f08cb80 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.223554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec3bad-5819-4246-a8cb-ca147f08cb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.252632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2dec3bad-5819-4246-a8cb-ca147f08cb80 None None] MFA Rules not processed for user `9939003319344ccb8cde9e9e8bab9649`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:14.253846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2dec3bad-5819-4246-a8cb-ca147f08cb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:14.272748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 639/1283] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:14 2022] PUT /identity/v3/projects/9b9bec5e97434a5a89f496d2349549df/users/601cc4a65cee4a86a421da37f6a980cb/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:14.273624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddbe74-9b2e-4707-9773-378bbd3688ba admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.288342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:14.343968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 645/1284] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:14.348356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec3bad-5819-4246-a8cb-ca147f08cb80 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.356029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddbe74-9b2e-4707-9773-378bbd3688ba admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.362737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7058f-c1df-4d69-852b-1dc0c07216a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.363301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7058f-c1df-4d69-852b-1dc0c07216a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.363879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7058f-c1df-4d69-852b-1dc0c07216a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.371174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7058f-c1df-4d69-852b-1dc0c07216a7 admin admin] PATH_INFO: `/v3/projects/6019625869c249fbae42eff10c82c590/users/495ed7e994004db99f1553ab6e0cc65c/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.388196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7058f-c1df-4d69-852b-1dc0c07216a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6019625869c249fbae42eff10c82c590, user_id=495ed7e994004db99f1553ab6e0cc65c, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.389210 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7058f-c1df-4d69-852b-1dc0c07216a7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.406084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec3bad-5819-4246-a8cb-ca147f08cb80 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.458254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d8c5-ae85-41ea-962d-58ef7e8a4491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.459208 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d8c5-ae85-41ea-962d-58ef7e8a4491 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.459346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d8c5-ae85-41ea-962d-58ef7e8a4491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.459622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d8c5-ae85-41ea-962d-58ef7e8a4491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.463017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 640/1285] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:14 2022] PUT /identity/v3/projects/6019625869c249fbae42eff10c82c590/users/495ed7e994004db99f1553ab6e0cc65c/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:14.467841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7058f-c1df-4d69-852b-1dc0c07216a7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.470681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308d8c5-ae85-41ea-962d-58ef7e8a4491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.472185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308d8c5-ae85-41ea-962d-58ef7e8a4491 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.476442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:14.484684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 646/1286] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:14.484791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbfd7-a950-47d7-ba9c-209eff2ff248 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.485025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbfd7-a950-47d7-ba9c-209eff2ff248 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.485298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbfd7-a950-47d7-ba9c-209eff2ff248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.504946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5d9bbfd7-a950-47d7-ba9c-209eff2ff248 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:14.506180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5d9bbfd7-a950-47d7-ba9c-209eff2ff248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:14.553445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7058f-c1df-4d69-852b-1dc0c07216a7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.558947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4a78c-3cc4-42fb-973c-67fbf28b5e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.559646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4a78c-3cc4-42fb-973c-67fbf28b5e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.560102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4a78c-3cc4-42fb-973c-67fbf28b5e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.560403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4a78c-3cc4-42fb-973c-67fbf28b5e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.561069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4a78c-3cc4-42fb-973c-67fbf28b5e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.562992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4a78c-3cc4-42fb-973c-67fbf28b5e0c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.578986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 647/1287] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:14.579933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270e6ed-0ca9-4bae-a9a1-19dad808769b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.580221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270e6ed-0ca9-4bae-a9a1-19dad808769b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.580540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270e6ed-0ca9-4bae-a9a1-19dad808769b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.612440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e270e6ed-0ca9-4bae-a9a1-19dad808769b None None] MFA Rules not processed for user `68fec4ef4a8549e5a4c5a79b296551a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:14.613637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e270e6ed-0ca9-4bae-a9a1-19dad808769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:14.640494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 641/1288] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:14.641898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab532e5c-9cc2-4cf2-ba10-42f52e8c8328 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.642247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab532e5c-9cc2-4cf2-ba10-42f52e8c8328 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.642589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab532e5c-9cc2-4cf2-ba10-42f52e8c8328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.673737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ab532e5c-9cc2-4cf2-ba10-42f52e8c8328 None None] MFA Rules not processed for user `601cc4a65cee4a86a421da37f6a980cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:14.676441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ab532e5c-9cc2-4cf2-ba10-42f52e8c8328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:14.692599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 648/1289] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:14.693866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf5030-8631-46f1-8f80-cf9fcfacb995 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.694152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf5030-8631-46f1-8f80-cf9fcfacb995 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.694429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf5030-8631-46f1-8f80-cf9fcfacb995 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.696734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 649/1290] 10.222.0.48 () {54 vars in 961 bytes} [Mon Oct 24 13:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:25:14.697920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c900ba-f271-4309-9522-3e0a1646ce15 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.698525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c900ba-f271-4309-9522-3e0a1646ce15 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.698819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c900ba-f271-4309-9522-3e0a1646ce15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.721446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a3c900ba-f271-4309-9522-3e0a1646ce15 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:14.722669 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a3c900ba-f271-4309-9522-3e0a1646ce15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:14.756704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 642/1291] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:14.759296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab532e5c-9cc2-4cf2-ba10-42f52e8c8328 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.766402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:14.800911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 650/1292] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:14.801587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c900ba-f271-4309-9522-3e0a1646ce15 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.811838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c900ba-f271-4309-9522-3e0a1646ce15 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.818454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98550a0e-660d-4b99-bab4-380e385208bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.819172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98550a0e-660d-4b99-bab4-380e385208bf admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.819459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98550a0e-660d-4b99-bab4-380e385208bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.819786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98550a0e-660d-4b99-bab4-380e385208bf admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.820474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98550a0e-660d-4b99-bab4-380e385208bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.823142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98550a0e-660d-4b99-bab4-380e385208bf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.910711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 651/1293] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:14.910711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98550a0e-660d-4b99-bab4-380e385208bf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:14.919774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98550a0e-660d-4b99-bab4-380e385208bf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.925253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2c83f-61f8-4232-aef6-c6a5ef06cf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.925871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c83f-61f8-4232-aef6-c6a5ef06cf6a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.926155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c83f-61f8-4232-aef6-c6a5ef06cf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.926424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2c83f-61f8-4232-aef6-c6a5ef06cf6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.927631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2c83f-61f8-4232-aef6-c6a5ef06cf6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.930028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2c83f-61f8-4232-aef6-c6a5ef06cf6a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.940868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:14.959340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab532e5c-9cc2-4cf2-ba10-42f52e8c8328 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:14.963837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b6d37-79d2-4938-af04-0a3ded0e81a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:14.964475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b6d37-79d2-4938-af04-0a3ded0e81a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.964748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b6d37-79d2-4938-af04-0a3ded0e81a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.965015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b6d37-79d2-4938-af04-0a3ded0e81a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:14.974442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b6d37-79d2-4938-af04-0a3ded0e81a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:14.976055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b6d37-79d2-4938-af04-0a3ded0e81a0 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:14.986514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 643/1294] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:14.987652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837eccf-6d37-410a-a25f-82aa1374264a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:14.987935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837eccf-6d37-410a-a25f-82aa1374264a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:14.988209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837eccf-6d37-410a-a25f-82aa1374264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.017869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 652/1295] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:15.019561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fcaff-37d9-4f0d-bd4f-17e1cd2a3d2c None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.020006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fcaff-37d9-4f0d-bd4f-17e1cd2a3d2c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.020467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fcaff-37d9-4f0d-bd4f-17e1cd2a3d2c None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.027438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 653/1296] 10.222.0.48 () {54 vars in 961 bytes} [Mon Oct 24 13:25:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:25:15.028655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fcaff-37d9-4f0d-bd4f-17e1cd2a3d2c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.031977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5837eccf-6d37-410a-a25f-82aa1374264a None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:15.033598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5837eccf-6d37-410a-a25f-82aa1374264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:15.099207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fcaff-37d9-4f0d-bd4f-17e1cd2a3d2c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.103392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ef1c7-cb50-44a9-aeb8-3553c85b43d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.104168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ef1c7-cb50-44a9-aeb8-3553c85b43d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.104619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ef1c7-cb50-44a9-aeb8-3553c85b43d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.104989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ef1c7-cb50-44a9-aeb8-3553c85b43d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.113135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 644/1297] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Oct 24 13:25:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:15.121772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837eccf-6d37-410a-a25f-82aa1374264a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.178712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ef1c7-cb50-44a9-aeb8-3553c85b43d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.180244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ef1c7-cb50-44a9-aeb8-3553c85b43d8 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.190040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 654/1298] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:15.194293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ef1c7-cb50-44a9-aeb8-3553c85b43d8 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.197952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837eccf-6d37-410a-a25f-82aa1374264a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.202545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde8714-d3f0-4c18-baf9-290bfce94812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.203291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8714-d3f0-4c18-baf9-290bfce94812 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.203574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8714-d3f0-4c18-baf9-290bfce94812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.203866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde8714-d3f0-4c18-baf9-290bfce94812 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.204549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8714-d3f0-4c18-baf9-290bfce94812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.206140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde8714-d3f0-4c18-baf9-290bfce94812 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.207990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ef1c7-cb50-44a9-aeb8-3553c85b43d8 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.217319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913471bc-86d9-4195-a5e9-d2cdd07138c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.217319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913471bc-86d9-4195-a5e9-d2cdd07138c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.217319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913471bc-86d9-4195-a5e9-d2cdd07138c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.217319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913471bc-86d9-4195-a5e9-d2cdd07138c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.217657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913471bc-86d9-4195-a5e9-d2cdd07138c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.217742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 645/1299] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:15.219246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913471bc-86d9-4195-a5e9-d2cdd07138c9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.221950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb368-69f1-48a1-9424-21d22e656580 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.222290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb368-69f1-48a1-9424-21d22e656580 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.222512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb368-69f1-48a1-9424-21d22e656580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.243934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0c8bb368-69f1-48a1-9424-21d22e656580 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:15.245178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c8bb368-69f1-48a1-9424-21d22e656580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:15.254945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 646/1300] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Oct 24 13:25:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:15.255756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb368-69f1-48a1-9424-21d22e656580 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.267891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb368-69f1-48a1-9424-21d22e656580 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.274529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e88a99-3132-4ac2-a4a1-a4898a3817ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.274529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88a99-3132-4ac2-a4a1-a4898a3817ab service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.274957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88a99-3132-4ac2-a4a1-a4898a3817ab service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.275342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e88a99-3132-4ac2-a4a1-a4898a3817ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.288154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:15.326504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 655/1301] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:15 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:15.327483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913471bc-86d9-4195-a5e9-d2cdd07138c9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.351354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913471bc-86d9-4195-a5e9-d2cdd07138c9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.362791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c54bd-eca8-4996-acd4-384ff01e1337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.366798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c54bd-eca8-4996-acd4-384ff01e1337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.366798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c54bd-eca8-4996-acd4-384ff01e1337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.366798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c54bd-eca8-4996-acd4-384ff01e1337 admin admin] PATH_INFO: `/v3/projects/862839b30f724620afc4fa49dc010774/users/bef6340b185f4f96a8a4538184062759/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.385400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e88a99-3132-4ac2-a4a1-a4898a3817ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.388879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e88a99-3132-4ac2-a4a1-a4898a3817ab service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.402994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c54bd-eca8-4996-acd4-384ff01e1337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862839b30f724620afc4fa49dc010774, user_id=bef6340b185f4f96a8a4538184062759, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.410406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c54bd-eca8-4996-acd4-384ff01e1337 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.414839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 647/1302] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:15.417396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e88a99-3132-4ac2-a4a1-a4898a3817ab service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.431671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e88a99-3132-4ac2-a4a1-a4898a3817ab service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.438334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cd662-9ea9-4d89-920b-c177214edd0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.440416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd662-9ea9-4d89-920b-c177214edd0f service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.440416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd662-9ea9-4d89-920b-c177214edd0f service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.440416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd662-9ea9-4d89-920b-c177214edd0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.514805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cd662-9ea9-4d89-920b-c177214edd0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.516961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cd662-9ea9-4d89-920b-c177214edd0f service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.528067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 656/1303] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:15 2022] PUT /identity/v3/projects/862839b30f724620afc4fa49dc010774/users/bef6340b185f4f96a8a4538184062759/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:15.529016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c54bd-eca8-4996-acd4-384ff01e1337 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.530537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 648/1304] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:15.532262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cd662-9ea9-4d89-920b-c177214edd0f service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.553659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cd662-9ea9-4d89-920b-c177214edd0f service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.663856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:15.669015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c54bd-eca8-4996-acd4-384ff01e1337 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.671390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c830d-422c-40ae-93a1-a5e3120950b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.672340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c830d-422c-40ae-93a1-a5e3120950b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.672696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c830d-422c-40ae-93a1-a5e3120950b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.673034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c830d-422c-40ae-93a1-a5e3120950b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.673780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c830d-422c-40ae-93a1-a5e3120950b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.675473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c830d-422c-40ae-93a1-a5e3120950b6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.678576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e0003-e8b6-40d2-9435-006c114f6dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.679211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0003-e8b6-40d2-9435-006c114f6dfe admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.679579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0003-e8b6-40d2-9435-006c114f6dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.679907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0003-e8b6-40d2-9435-006c114f6dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.680845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e0003-e8b6-40d2-9435-006c114f6dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.684450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e0003-e8b6-40d2-9435-006c114f6dfe admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.690845 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 649/1305] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:15.691896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c830d-422c-40ae-93a1-a5e3120950b6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.703315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 657/1306] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:15.703716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c830d-422c-40ae-93a1-a5e3120950b6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.704918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e0003-e8b6-40d2-9435-006c114f6dfe admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:15.708904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9b571-bf5d-4b31-9eee-ea76201f6936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.713901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9b571-bf5d-4b31-9eee-ea76201f6936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.713901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9b571-bf5d-4b31-9eee-ea76201f6936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.713901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9b571-bf5d-4b31-9eee-ea76201f6936 admin admin] PATH_INFO: `/v3/projects/862839b30f724620afc4fa49dc010774/users/bef6340b185f4f96a8a4538184062759/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.713901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9b571-bf5d-4b31-9eee-ea76201f6936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862839b30f724620afc4fa49dc010774, user_id=bef6340b185f4f96a8a4538184062759, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.715388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9b571-bf5d-4b31-9eee-ea76201f6936 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.718436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e0003-e8b6-40d2-9435-006c114f6dfe admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:15.724952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297c6d5-9627-4bfd-bf1b-1fbe486c2b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:15.726592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297c6d5-9627-4bfd-bf1b-1fbe486c2b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.727010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297c6d5-9627-4bfd-bf1b-1fbe486c2b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.727413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297c6d5-9627-4bfd-bf1b-1fbe486c2b97 admin admin] PATH_INFO: `/v3/projects/0b54c486ea1c4ce0b28b5e5a82da7886/users/eb817fd572aa4e3d968504c110acf1af/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.731360 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 650/1307] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:15 2022] PUT /identity/v3/projects/862839b30f724620afc4fa49dc010774/users/bef6340b185f4f96a8a4538184062759/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:15.732818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b6811-36e0-47fa-b844-66eaf56d842c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.737455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b6811-36e0-47fa-b844-66eaf56d842c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.737455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b6811-36e0-47fa-b844-66eaf56d842c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.737455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 651/1308] 10.222.0.48 () {54 vars in 912 bytes} [Mon Oct 24 13:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:25:15.743688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2abc5-33b7-439e-a6cb-955d8e89d059 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.744637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2abc5-33b7-439e-a6cb-955d8e89d059 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.745824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2abc5-33b7-439e-a6cb-955d8e89d059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.760248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297c6d5-9627-4bfd-bf1b-1fbe486c2b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b54c486ea1c4ce0b28b5e5a82da7886, user_id=eb817fd572aa4e3d968504c110acf1af, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:15.767503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297c6d5-9627-4bfd-bf1b-1fbe486c2b97 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:15.790127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-def2abc5-33b7-439e-a6cb-955d8e89d059 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:15.791497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-def2abc5-33b7-439e-a6cb-955d8e89d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:15.830238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:15.846339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 658/1309] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:15 2022] PUT /identity/v3/projects/0b54c486ea1c4ce0b28b5e5a82da7886/users/eb817fd572aa4e3d968504c110acf1af/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:15.850111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d410cc-ae70-4312-a5be-4fb964db0c5c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:15.850111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d410cc-ae70-4312-a5be-4fb964db0c5c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:15.850111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d410cc-ae70-4312-a5be-4fb964db0c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:15.920941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e4d410cc-ae70-4312-a5be-4fb964db0c5c None None] MFA Rules not processed for user `bef6340b185f4f96a8a4538184062759`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:15.920941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e4d410cc-ae70-4312-a5be-4fb964db0c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:15.923130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 652/1310] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:25:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:15.924594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2abc5-33b7-439e-a6cb-955d8e89d059 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:16.025789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2abc5-33b7-439e-a6cb-955d8e89d059 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:16.029410 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 659/1311] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:16.030694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d410cc-ae70-4312-a5be-4fb964db0c5c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:16.141912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247550c-ebd6-44ea-b887-67bba18eba24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:16.142511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247550c-ebd6-44ea-b887-67bba18eba24 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.142790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247550c-ebd6-44ea-b887-67bba18eba24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.143060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247550c-ebd6-44ea-b887-67bba18eba24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.146821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247550c-ebd6-44ea-b887-67bba18eba24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:16.156933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247550c-ebd6-44ea-b887-67bba18eba24 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:16.157959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d410cc-ae70-4312-a5be-4fb964db0c5c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:16.162534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76df530-84a1-474f-bce8-f406637e7c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:16.163262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76df530-84a1-474f-bce8-f406637e7c96 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.163711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76df530-84a1-474f-bce8-f406637e7c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.164253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76df530-84a1-474f-bce8-f406637e7c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.177419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 653/1312] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:16.178550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247550c-ebd6-44ea-b887-67bba18eba24 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:16.189509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247550c-ebd6-44ea-b887-67bba18eba24 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:16.193276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e5ce8-c21a-4949-b0cb-78af80b9676a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:16.193889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e5ce8-c21a-4949-b0cb-78af80b9676a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.194163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e5ce8-c21a-4949-b0cb-78af80b9676a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.194430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e5ce8-c21a-4949-b0cb-78af80b9676a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.205686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e5ce8-c21a-4949-b0cb-78af80b9676a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:16.207780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e5ce8-c21a-4949-b0cb-78af80b9676a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:16.231940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 654/1313] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:16.232982 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a6a4f-99c2-4bd7-b170-b29725015563 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.233265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a6a4f-99c2-4bd7-b170-b29725015563 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.233538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a6a4f-99c2-4bd7-b170-b29725015563 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.246141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 655/1314] 10.222.0.48 () {54 vars in 912 bytes} [Mon Oct 24 13:25:16 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:25:16.247123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a6a4f-99c2-4bd7-b170-b29725015563 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:16.264098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:16.267660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76df530-84a1-474f-bce8-f406637e7c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:16.272897 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76df530-84a1-474f-bce8-f406637e7c96 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:16.276398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a6a4f-99c2-4bd7-b170-b29725015563 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:16.285608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281be65-027b-4e5c-9cc1-0380feb72e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:16.286451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 660/1315] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:16.288309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a938e-d53d-4479-9f8f-124771f1e600 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.290115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a938e-d53d-4479-9f8f-124771f1e600 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.290115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a938e-d53d-4479-9f8f-124771f1e600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.300315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281be65-027b-4e5c-9cc1-0380feb72e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.300944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281be65-027b-4e5c-9cc1-0380feb72e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.301226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281be65-027b-4e5c-9cc1-0380feb72e43 admin admin] PATH_INFO: `/v3/projects/0b54c486ea1c4ce0b28b5e5a82da7886/users/eb817fd572aa4e3d968504c110acf1af/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.310927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281be65-027b-4e5c-9cc1-0380feb72e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b54c486ea1c4ce0b28b5e5a82da7886, user_id=eb817fd572aa4e3d968504c110acf1af, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:16.322823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281be65-027b-4e5c-9cc1-0380feb72e43 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:16.334192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ec3a938e-d53d-4479-9f8f-124771f1e600 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:16.336343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ec3a938e-d53d-4479-9f8f-124771f1e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:16.366956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 661/1316] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:16.373389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 656/1317] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:16 2022] PUT /identity/v3/projects/0b54c486ea1c4ce0b28b5e5a82da7886/users/eb817fd572aa4e3d968504c110acf1af/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:16.384948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeda14-a13a-4f78-b7d5-3cd1b7c7d6f1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.386248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeda14-a13a-4f78-b7d5-3cd1b7c7d6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.386682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbeda14-a13a-4f78-b7d5-3cd1b7c7d6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.433818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281be65-027b-4e5c-9cc1-0380feb72e43 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:16.447588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3fbeda14-a13a-4f78-b7d5-3cd1b7c7d6f1 None None] MFA Rules not processed for user `eb817fd572aa4e3d968504c110acf1af`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:16.449635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3fbeda14-a13a-4f78-b7d5-3cd1b7c7d6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:16.520302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:16.532102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281be65-027b-4e5c-9cc1-0380feb72e43 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:16.545835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b3ba5-815b-4a0a-b232-2a650c5b9779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:16.545835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b3ba5-815b-4a0a-b232-2a650c5b9779 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.545835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b3ba5-815b-4a0a-b232-2a650c5b9779 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.545835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b3ba5-815b-4a0a-b232-2a650c5b9779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.610619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 662/1318] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:16.668459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b3ba5-815b-4a0a-b232-2a650c5b9779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:16.670012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b3ba5-815b-4a0a-b232-2a650c5b9779 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:16.684104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 657/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:25:16.884562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbeda14-a13a-4f78-b7d5-3cd1b7c7d6f1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:16.905901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbeda14-a13a-4f78-b7d5-3cd1b7c7d6f1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:16.910230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04d7c-eaa1-41ec-be3f-a6079b91d43c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:16.910640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04d7c-eaa1-41ec-be3f-a6079b91d43c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:16.910878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04d7c-eaa1-41ec-be3f-a6079b91d43c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:16.911297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04d7c-eaa1-41ec-be3f-a6079b91d43c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:16.922941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b04d7c-eaa1-41ec-be3f-a6079b91d43c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:16.924580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b04d7c-eaa1-41ec-be3f-a6079b91d43c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:16.934580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 663/1320] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:17.567945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b3ba5-815b-4a0a-b232-2a650c5b9779 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:17.579543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b3ba5-815b-4a0a-b232-2a650c5b9779 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:17.585999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaaf41-0224-4b00-928d-a628eade676e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:17.587539 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaaf41-0224-4b00-928d-a628eade676e service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:17.588105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaaf41-0224-4b00-928d-a628eade676e service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:17.588721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaaf41-0224-4b00-928d-a628eade676e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:17.733960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaaf41-0224-4b00-928d-a628eade676e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:17.735743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaaf41-0224-4b00-928d-a628eade676e service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:17.750755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 658/1321] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:17.951906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04d7c-eaa1-41ec-be3f-a6079b91d43c service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:18.113087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04d7c-eaa1-41ec-be3f-a6079b91d43c service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:18.139503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:18.139503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:18.139503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:18.139503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] PATH_INFO: `/v3/users/601cc4a65cee4a86a421da37f6a980cb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:18.140505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] RBAC: Authorizing `identity:delete_user(user_id=601cc4a65cee4a86a421da37f6a980cb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:18.140505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: WARNING py.warnings [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:25:18.140505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: warnings.warn(msg) Oct 24 13:25:18.140505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: Oct 24 13:25:18.141042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaaf41-0224-4b00-928d-a628eade676e service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:18.143167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:18.293290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaaf41-0224-4b00-928d-a628eade676e service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:18.309064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef6340b185f4f96a8a4538184062759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '862839b30f724620afc4fa49dc010774', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:18.312485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:18.312485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:18.312485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:18.313101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:18.313808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:18.317307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:18.344262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 659/1322] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:18.348934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:18.372961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf181901-352a-4bc1-a78f-56fe93ad33af tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:18.378491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef6340b185f4f96a8a4538184062759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '862839b30f724620afc4fa49dc010774', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:18.379411 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:18.379758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:18.380037 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:18.380772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:18.382971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:18.406508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:18.446156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601cc4a65cee4a86a421da37f6a980cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:18.447192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601cc4a65cee4a86a421da37f6a980cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:18.481920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:18.492827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601cc4a65cee4a86a421da37f6a980cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:18.504107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601cc4a65cee4a86a421da37f6a980cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:18.505102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601cc4a65cee4a86a421da37f6a980cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:18.547180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 660/1323] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:18 2022] POST /identity/v3/projects => generated 354 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:18.551314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:18.553370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:18.699240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eef1f-2b42-459b-be85-e70afb24902b tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:18.705868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06effd4-d572-4aa5-85d3-8114307f957d tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef6340b185f4f96a8a4538184062759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '862839b30f724620afc4fa49dc010774', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:18.706645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06effd4-d572-4aa5-85d3-8114307f957d tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:18.706938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06effd4-d572-4aa5-85d3-8114307f957d tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:18.707479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06effd4-d572-4aa5-85d3-8114307f957d tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:18.708143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06effd4-d572-4aa5-85d3-8114307f957d tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:18.708474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06effd4-d572-4aa5-85d3-8114307f957d tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:18.710734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06effd4-d572-4aa5-85d3-8114307f957d tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:18.726129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 661/1324] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:25:18.742467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 664/1325] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:17 2022] DELETE /identity/v3/users/601cc4a65cee4a86a421da37f6a980cb => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:18.746521 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74750523-bfde-469c-8213-a2125990bc6a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:18.747986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74750523-bfde-469c-8213-a2125990bc6a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:18.747986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74750523-bfde-469c-8213-a2125990bc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:18.801060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:18.806157 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-74750523-bfde-469c-8213-a2125990bc6a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:18.806569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-74750523-bfde-469c-8213-a2125990bc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:18.889075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 662/1326] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:18.923085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74750523-bfde-469c-8213-a2125990bc6a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:18.925375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-74750523-bfde-469c-8213-a2125990bc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:18.966922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18acb9a-fcc2-4c2d-8fcc-360951a2346d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:18.975828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef6340b185f4f96a8a4538184062759', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '862839b30f724620afc4fa49dc010774', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:18.978692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:18.978993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:18.980521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] PATH_INFO: `/v3/projects/8063aa441c104ea5bca6a343b7754aa7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:18.984377 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8063aa441c104ea5bca6a343b7754aa7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:18.987844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:19.067725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74750523-bfde-469c-8213-a2125990bc6a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:19.073566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:19.074014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:19.074421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:19.074759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:19.096216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:19.099019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:19.129234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 663/1327] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:19.134880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8063aa441c104ea5bca6a343b7754aa7', 'request_id': 'req-772d4c7c-b140-44f0-982e-02a5130a4640'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:19.204289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8063aa441c104ea5bca6a343b7754aa7', 'request_id': 'req-772d4c7c-b140-44f0-982e-02a5130a4640'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:19.212248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8063aa441c104ea5bca6a343b7754aa7', 'request_id': 'req-772d4c7c-b140-44f0-982e-02a5130a4640'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:19.237316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] The token cache is being invalidate because project 8063aa441c104ea5bca6a343b7754aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:19.237618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:19.240114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:19.244861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 665/1328] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:18 2022] DELETE /identity/v3/projects/8063aa441c104ea5bca6a343b7754aa7 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:20.132391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:20.138731 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:20.159527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225430b-cbe2-4058-bde9-c8b31c71e9a8 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:20.293724 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:20.330919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:20.333872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:20.334194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:20.334473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] PATH_INFO: `/v3/users/706e3bfc697d4073b8f9cf6c98f98147` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:20.337233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=706e3bfc697d4073b8f9cf6c98f98147)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:20.343572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:20.659797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706e3bfc697d4073b8f9cf6c98f98147'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:20.685826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706e3bfc697d4073b8f9cf6c98f98147'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:20.740704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706e3bfc697d4073b8f9cf6c98f98147'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:20.872202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706e3bfc697d4073b8f9cf6c98f98147'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:20.882784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-58f31af4-bb61-45ff-b7d5-a67ca5514397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706e3bfc697d4073b8f9cf6c98f98147'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:20.882784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 664/1329] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:20 2022] DELETE /identity/v3/users/706e3bfc697d4073b8f9cf6c98f98147 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:20.885606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:20.887689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:20.891949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de100-d6dc-4620-af59-47c17d028f8a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:20.897244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de100-d6dc-4620-af59-47c17d028f8a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:20.897536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de100-d6dc-4620-af59-47c17d028f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:20.934469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d4c7c-b140-44f0-982e-02a5130a4640 tempest-VolumeQuotaClassesTest-862361816 tempest-VolumeQuotaClassesTest-862361816-project-admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:20.964887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8e8de100-d6dc-4620-af59-47c17d028f8a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:20.965892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e8de100-d6dc-4620-af59-47c17d028f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:21.167641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 665/1330] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:21.169088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:21.169758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:21.170044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:21.170318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] PATH_INFO: `/v3/projects/9b9bec5e97434a5a89f496d2349549df` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:21.172987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9bec5e97434a5a89f496d2349549df)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:21.174943 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:21.240555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:21.322125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de100-d6dc-4620-af59-47c17d028f8a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:21.324103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e8de100-d6dc-4620-af59-47c17d028f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:21.403101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9bec5e97434a5a89f496d2349549df', 'request_id': 'req-8d203a34-8986-4e3b-acb3-e893e18b31f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:21.507748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de100-d6dc-4620-af59-47c17d028f8a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:21.524026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9bec5e97434a5a89f496d2349549df', 'request_id': 'req-8d203a34-8986-4e3b-acb3-e893e18b31f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:21.524586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:21.532038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9bec5e97434a5a89f496d2349549df', 'request_id': 'req-8d203a34-8986-4e3b-acb3-e893e18b31f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:21.559289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] The token cache is being invalidate because project 9b9bec5e97434a5a89f496d2349549df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:21.559866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:21.569991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8d203a34-8986-4e3b-acb3-e893e18b31f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:21.569991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 666/1331] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:20 2022] DELETE /identity/v3/projects/9b9bec5e97434a5a89f496d2349549df => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:21.582179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157f32e-0c86-4a12-8e03-b0f0ab8560db None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:21.582571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157f32e-0c86-4a12-8e03-b0f0ab8560db None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:21.583009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157f32e-0c86-4a12-8e03-b0f0ab8560db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:21.632723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:21.633909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:21.634693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:21.635296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:21.638587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:21.643106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7157f32e-0c86-4a12-8e03-b0f0ab8560db None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:21.650315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7157f32e-0c86-4a12-8e03-b0f0ab8560db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:21.791072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:21.810125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:21.815332 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 667/1332] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:21.823774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207aec6c-a447-4304-a5b3-e218f4de2183 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:21.823774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207aec6c-a447-4304-a5b3-e218f4de2183 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:21.823774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207aec6c-a447-4304-a5b3-e218f4de2183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:21.834416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 666/1333] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:25:21.851432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-207aec6c-a447-4304-a5b3-e218f4de2183 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:21.853223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-207aec6c-a447-4304-a5b3-e218f4de2183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:21.865383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 668/1334] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:21.870619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:21.881776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096b807-6522-4a2c-9878-35589edb6fea service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:21.892501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:21.893151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:21.893561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:21.898396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:21.898396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:21.898396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:21.898396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:21.914172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207aec6c-a447-4304-a5b3-e218f4de2183 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:21.927595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 667/1335] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:21.927711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207aec6c-a447-4304-a5b3-e218f4de2183 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:21.931130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb51d2c-ff38-460d-a677-6e130c75ba6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:21.931845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb51d2c-ff38-460d-a677-6e130c75ba6b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:21.932200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb51d2c-ff38-460d-a677-6e130c75ba6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:21.932611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb51d2c-ff38-460d-a677-6e130c75ba6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:21.933472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb51d2c-ff38-460d-a677-6e130c75ba6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:21.935676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb51d2c-ff38-460d-a677-6e130c75ba6b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:22.073236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:22.099487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 669/1336] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:21 2022] POST /identity/v3/projects => generated 403 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:22.101691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61eb00-ab0b-4538-b263-8f24607fe6b9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:22.261103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294063e1-c067-40d8-a284-42dcf8e45159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:22.261756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294063e1-c067-40d8-a284-42dcf8e45159 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:22.266375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294063e1-c067-40d8-a284-42dcf8e45159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:22.266841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294063e1-c067-40d8-a284-42dcf8e45159 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:22.267679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294063e1-c067-40d8-a284-42dcf8e45159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:22.273353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294063e1-c067-40d8-a284-42dcf8e45159 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:22.450456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 668/1337] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:22 2022] POST /identity/v3/users => generated 302 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:22.454905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb51d2c-ff38-460d-a677-6e130c75ba6b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:22.470472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb51d2c-ff38-460d-a677-6e130c75ba6b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:22.485313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac701e-46ca-4575-975c-4839338c2965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:22.488596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac701e-46ca-4575-975c-4839338c2965 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:22.488596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac701e-46ca-4575-975c-4839338c2965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:22.488596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac701e-46ca-4575-975c-4839338c2965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:22.488596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac701e-46ca-4575-975c-4839338c2965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:22.489776 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac701e-46ca-4575-975c-4839338c2965 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:22.519140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294063e1-c067-40d8-a284-42dcf8e45159 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:22.529794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 670/1338] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:22.548117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294063e1-c067-40d8-a284-42dcf8e45159 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:22.569311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3ffbd-6098-419c-95f5-3e5fbaae29de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:22.570827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3ffbd-6098-419c-95f5-3e5fbaae29de admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:22.571374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3ffbd-6098-419c-95f5-3e5fbaae29de admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:22.571862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3ffbd-6098-419c-95f5-3e5fbaae29de admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:22.574182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3ffbd-6098-419c-95f5-3e5fbaae29de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:22.576083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3ffbd-6098-419c-95f5-3e5fbaae29de admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:22.616667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac701e-46ca-4575-975c-4839338c2965 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:22.622439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 669/1339] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:25:22.628872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac701e-46ca-4575-975c-4839338c2965 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:22.653703 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd8d5c-73f9-49c8-aff3-4c1933f40047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:22.654482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd8d5c-73f9-49c8-aff3-4c1933f40047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:22.654828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd8d5c-73f9-49c8-aff3-4c1933f40047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:22.655119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd8d5c-73f9-49c8-aff3-4c1933f40047 admin admin] PATH_INFO: `/v3/projects/e26b6fb4664948ffbcc626f8ef0124e9/users/0d212915d1674b3986b98e1ba304d845/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:22.728847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd8d5c-73f9-49c8-aff3-4c1933f40047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26b6fb4664948ffbcc626f8ef0124e9, user_id=0d212915d1674b3986b98e1ba304d845, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:22.736559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd8d5c-73f9-49c8-aff3-4c1933f40047 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:22.851800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 671/1340] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:22 2022] PUT /identity/v3/projects/e26b6fb4664948ffbcc626f8ef0124e9/users/0d212915d1674b3986b98e1ba304d845/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:22.868757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdae4f1-e0d6-4aea-85d7-ee8cd4b345be None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:22.869107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdae4f1-e0d6-4aea-85d7-ee8cd4b345be None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:22.869788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdae4f1-e0d6-4aea-85d7-ee8cd4b345be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:22.944879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fbdae4f1-e0d6-4aea-85d7-ee8cd4b345be None None] MFA Rules not processed for user `0d212915d1674b3986b98e1ba304d845`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:22.946705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fbdae4f1-e0d6-4aea-85d7-ee8cd4b345be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:23.086120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd8d5c-73f9-49c8-aff3-4c1933f40047 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:23.092800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 670/1341] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:23.225478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:23.276513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd8d5c-73f9-49c8-aff3-4c1933f40047 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:23.293514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a563fa-7a5e-4c13-bf6e-27c5f81ba8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:23.294153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a563fa-7a5e-4c13-bf6e-27c5f81ba8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:23.294511 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a563fa-7a5e-4c13-bf6e-27c5f81ba8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:23.294689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a563fa-7a5e-4c13-bf6e-27c5f81ba8e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:23.295426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a563fa-7a5e-4c13-bf6e-27c5f81ba8e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:23.313227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a563fa-7a5e-4c13-bf6e-27c5f81ba8e4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:23.473059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 672/1342] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:23 2022] POST /identity/v3/projects => generated 403 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:23.477717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdae4f1-e0d6-4aea-85d7-ee8cd4b345be None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:23.508741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdae4f1-e0d6-4aea-85d7-ee8cd4b345be None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:23.589247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:23.636688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9fdd3-c381-48af-b1cc-16231040963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:23.643668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9fdd3-c381-48af-b1cc-16231040963e admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:23.644021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9fdd3-c381-48af-b1cc-16231040963e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:23.644326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9fdd3-c381-48af-b1cc-16231040963e admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:23.645006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9fdd3-c381-48af-b1cc-16231040963e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:23.650429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9fdd3-c381-48af-b1cc-16231040963e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:23.771666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 671/1343] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:23 2022] POST /identity/v3/users => generated 308 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:23.776046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a563fa-7a5e-4c13-bf6e-27c5f81ba8e4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:23.792832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a563fa-7a5e-4c13-bf6e-27c5f81ba8e4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:23.805194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:23.805896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:23.806192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:23.806443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:23.807094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:23.808756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:23.840114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 673/1344] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:23.844247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9fdd3-c381-48af-b1cc-16231040963e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:23.859999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9fdd3-c381-48af-b1cc-16231040963e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:23.889902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf1224-d843-49af-b9dd-f35748ca296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:23.890969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1224-d843-49af-b9dd-f35748ca296a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:23.891362 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1224-d843-49af-b9dd-f35748ca296a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:23.891745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1224-d843-49af-b9dd-f35748ca296a admin admin] PATH_INFO: `/v3/projects/c01e766de16b4abbb1867414c7ee424e/users/26ee4461d90549ca9e44996b1f66545b/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:23.958356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf1224-d843-49af-b9dd-f35748ca296a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01e766de16b4abbb1867414c7ee424e, user_id=26ee4461d90549ca9e44996b1f66545b, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:23.966720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf1224-d843-49af-b9dd-f35748ca296a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:24.045124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:24.047352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:24.080147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 672/1345] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:23 2022] PUT /identity/v3/projects/c01e766de16b4abbb1867414c7ee424e/users/26ee4461d90549ca9e44996b1f66545b/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:24.088911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf1224-d843-49af-b9dd-f35748ca296a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:24.112876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363fdc-f259-406d-a5ed-3b52009b5f24 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:24.287113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf1224-d843-49af-b9dd-f35748ca296a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:24.300778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b33fe-0de6-4ba8-84c5-5c448376cdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:24.302047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b33fe-0de6-4ba8-84c5-5c448376cdff admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:24.302966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b33fe-0de6-4ba8-84c5-5c448376cdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:24.304119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b33fe-0de6-4ba8-84c5-5c448376cdff admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:24.306434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b33fe-0de6-4ba8-84c5-5c448376cdff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:24.309655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b33fe-0de6-4ba8-84c5-5c448376cdff admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:24.341971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 673/1346] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:25:24.346922 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b33fe-0de6-4ba8-84c5-5c448376cdff admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:24.359490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:24.363119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:24.365080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:24.366800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] PATH_INFO: `/v3/projects/60e0c5c137ef4274a8c6ee990b4d8296` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:24.387433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e0c5c137ef4274a8c6ee990b4d8296)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:24.393765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:24.409829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b33fe-0de6-4ba8-84c5-5c448376cdff admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:24.425639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d865fe2-ad78-462b-888c-fd15c8f0dedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:24.426734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d865fe2-ad78-462b-888c-fd15c8f0dedc admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:24.427235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d865fe2-ad78-462b-888c-fd15c8f0dedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:24.429843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d865fe2-ad78-462b-888c-fd15c8f0dedc admin admin] PATH_INFO: `/v3/projects/c01e766de16b4abbb1867414c7ee424e/users/26ee4461d90549ca9e44996b1f66545b/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:24.441871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d865fe2-ad78-462b-888c-fd15c8f0dedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01e766de16b4abbb1867414c7ee424e, user_id=26ee4461d90549ca9e44996b1f66545b, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:24.453423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d865fe2-ad78-462b-888c-fd15c8f0dedc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:24.495842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 674/1347] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:24 2022] PUT /identity/v3/projects/c01e766de16b4abbb1867414c7ee424e/users/26ee4461d90549ca9e44996b1f66545b/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:24.506224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3c48-1ed1-4e27-b814-e625fe5a0339 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:24.507377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3c48-1ed1-4e27-b814-e625fe5a0339 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:24.507951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3c48-1ed1-4e27-b814-e625fe5a0339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:24.551735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-73cc3c48-1ed1-4e27-b814-e625fe5a0339 None None] MFA Rules not processed for user `26ee4461d90549ca9e44996b1f66545b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:24.553063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-73cc3c48-1ed1-4e27-b814-e625fe5a0339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:24.631767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e0c5c137ef4274a8c6ee990b4d8296', 'request_id': 'req-50bd6959-c40f-425b-9cc2-482310f60b36'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:24.708138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 675/1348] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:24 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:25:24.714288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e0c5c137ef4274a8c6ee990b4d8296', 'request_id': 'req-50bd6959-c40f-425b-9cc2-482310f60b36'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:24.728214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e0c5c137ef4274a8c6ee990b4d8296', 'request_id': 'req-50bd6959-c40f-425b-9cc2-482310f60b36'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:24.735668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc3c48-1ed1-4e27-b814-e625fe5a0339 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:24.737121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-73cc3c48-1ed1-4e27-b814-e625fe5a0339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:24.768096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] The token cache is being invalidate because project 60e0c5c137ef4274a8c6ee990b4d8296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:24.769157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:24.771457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:24.774933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 674/1349] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:24 2022] DELETE /identity/v3/projects/60e0c5c137ef4274a8c6ee990b4d8296 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:24.779983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:24.782695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:24.830205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:24.883883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc3c48-1ed1-4e27-b814-e625fe5a0339 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:24.970529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd6959-c40f-425b-9cc2-482310f60b36 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:24.991974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:24.993509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:24.994988 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:24.995872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] PATH_INFO: `/v3/users/bef6340b185f4f96a8a4538184062759` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:24.999960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:25.003528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:25.003706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef6340b185f4f96a8a4538184062759)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:25.003814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:25.003814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:25.003992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:25.010008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:25.067448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:25.159471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:25.160927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:25.183171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 676/1350] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:24 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:25.257835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef6340b185f4f96a8a4538184062759'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:25.260042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef6340b185f4f96a8a4538184062759'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:25.288666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef6340b185f4f96a8a4538184062759'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:25.318010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef6340b185f4f96a8a4538184062759'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:25.321802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef6340b185f4f96a8a4538184062759'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:25.443801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 675/1351] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:24 2022] DELETE /identity/v3/users/bef6340b185f4f96a8a4538184062759 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:25.712016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:25.714290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:25.740844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51164bb8-f4a0-46df-9873-6d6419e0eac2 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:25.854033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:25.854033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:25.854033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:25.854754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] PATH_INFO: `/v3/users/68fec4ef4a8549e5a4c5a79b296551a5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:25.857813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68fec4ef4a8549e5a4c5a79b296551a5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:25.864003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:26.009631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:26.011112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:26.043547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e314d9-efef-4684-a107-a4f9354f221a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:26.060406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:26.066527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:26.066527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:26.066527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] PATH_INFO: `/v3/users/0d212915d1674b3986b98e1ba304d845` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:26.066527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d212915d1674b3986b98e1ba304d845)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:26.069410 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:26.124309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:26.144455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68fec4ef4a8549e5a4c5a79b296551a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.165450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68fec4ef4a8549e5a4c5a79b296551a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.207454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68fec4ef4a8549e5a4c5a79b296551a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.316173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d212915d1674b3986b98e1ba304d845'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.317265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d212915d1674b3986b98e1ba304d845'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.324447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68fec4ef4a8549e5a4c5a79b296551a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.325097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb1b5728-0f96-4646-9713-4b3393094135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68fec4ef4a8549e5a4c5a79b296551a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.340903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 677/1352] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:25 2022] DELETE /identity/v3/users/68fec4ef4a8549e5a4c5a79b296551a5 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:26.342457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322cd8-e410-4db2-8e38-3b3663f45828 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:26.342741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322cd8-e410-4db2-8e38-3b3663f45828 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:26.343015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322cd8-e410-4db2-8e38-3b3663f45828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:26.350046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d212915d1674b3986b98e1ba304d845'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.371609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d212915d1674b3986b98e1ba304d845'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.374574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84972b30-8dee-4efb-93b9-047a197f5fb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d212915d1674b3986b98e1ba304d845'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:26.393598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0c322cd8-e410-4db2-8e38-3b3663f45828 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:26.396683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c322cd8-e410-4db2-8e38-3b3663f45828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:26.480108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 676/1353] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:26 2022] DELETE /identity/v3/users/0d212915d1674b3986b98e1ba304d845 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:26.490953 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495e3a4-622d-4abc-a4f3-4daeaaa09c02 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:26.490953 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495e3a4-622d-4abc-a4f3-4daeaaa09c02 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:26.490953 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495e3a4-622d-4abc-a4f3-4daeaaa09c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:26.511407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8495e3a4-622d-4abc-a4f3-4daeaaa09c02 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:26.513174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8495e3a4-622d-4abc-a4f3-4daeaaa09c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:26.575452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 678/1354] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:26.597644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c322cd8-e410-4db2-8e38-3b3663f45828 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:26.600219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c322cd8-e410-4db2-8e38-3b3663f45828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:26.658947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 677/1355] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:25:26.683538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c322cd8-e410-4db2-8e38-3b3663f45828 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:26.696518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495e3a4-622d-4abc-a4f3-4daeaaa09c02 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:26.698800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8495e3a4-622d-4abc-a4f3-4daeaaa09c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:26.750345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:26.751126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:26.751519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:26.751946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:26.763007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:26.765237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:26.780239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 679/1356] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:26.798419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:26.838679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495e3a4-622d-4abc-a4f3-4daeaaa09c02 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:26.848976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca298a7-ce36-49ce-8020-8939b82b397d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:26.849665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca298a7-ce36-49ce-8020-8939b82b397d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:26.849881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca298a7-ce36-49ce-8020-8939b82b397d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:26.850152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca298a7-ce36-49ce-8020-8939b82b397d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:26.882335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca298a7-ce36-49ce-8020-8939b82b397d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:26.884398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca298a7-ce36-49ce-8020-8939b82b397d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:26.917775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 678/1357] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:25:26.959521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:27.685110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:27.687354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:27.709582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80d300-e084-4a8a-b697-a24ce16e7260 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:27.731164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:27.732195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:27.732733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:27.733228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] PATH_INFO: `/v3/projects/862839b30f724620afc4fa49dc010774` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:27.734787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] RBAC: Authorizing `identity:delete_project(project_id=862839b30f724620afc4fa49dc010774)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:27.736108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:25:27.736108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Oct 24 13:25:27.736108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: Oct 24 13:25:27.739348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:27.951600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862839b30f724620afc4fa49dc010774', 'request_id': 'req-c84b9926-0dc2-4118-8b1c-91bb80eadcee'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:27.971523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862839b30f724620afc4fa49dc010774', 'request_id': 'req-c84b9926-0dc2-4118-8b1c-91bb80eadcee'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:27.993735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862839b30f724620afc4fa49dc010774', 'request_id': 'req-c84b9926-0dc2-4118-8b1c-91bb80eadcee'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:28.062477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247daf9-8012-4185-a7aa-68f7287ad4af None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:28.063595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247daf9-8012-4185-a7aa-68f7287ad4af None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:28.064235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247daf9-8012-4185-a7aa-68f7287ad4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:28.101139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] The token cache is being invalidate because project 862839b30f724620afc4fa49dc010774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:28.101605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:28.102959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c84b9926-0dc2-4118-8b1c-91bb80eadcee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:28.107199 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 680/1358] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:27 2022] DELETE /identity/v3/projects/862839b30f724620afc4fa49dc010774 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:28.113293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b78fa7-f5d7-4692-be1d-925e667e9477 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:28.120187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b78fa7-f5d7-4692-be1d-925e667e9477 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:28.120187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b78fa7-f5d7-4692-be1d-925e667e9477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:28.149295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b247daf9-8012-4185-a7aa-68f7287ad4af None None] MFA Rules not processed for user `7c096064e6ec443a9e8755bc2e68dbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:28.150568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b247daf9-8012-4185-a7aa-68f7287ad4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:28.169794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-84b78fa7-f5d7-4692-be1d-925e667e9477 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:28.175809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-84b78fa7-f5d7-4692-be1d-925e667e9477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:28.335760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:28.338585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 679/1359] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:28.344148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247daf9-8012-4185-a7aa-68f7287ad4af None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:28.346921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b247daf9-8012-4185-a7aa-68f7287ad4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:28.360916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 681/1360] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:28.393574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b0764-c945-4e42-a8aa-a912dc65da71 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:28.396458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b0764-c945-4e42-a8aa-a912dc65da71 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:28.398613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b0764-c945-4e42-a8aa-a912dc65da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:28.450856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-796b0764-c945-4e42-a8aa-a912dc65da71 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:28.453938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-796b0764-c945-4e42-a8aa-a912dc65da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:28.474820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 682/1361] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:28.478863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b0764-c945-4e42-a8aa-a912dc65da71 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:28.499322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b0764-c945-4e42-a8aa-a912dc65da71 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:28.516962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:28.517589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:28.517872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:28.518146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:28.518794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:28.519098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:28.520762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:28.530844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:28.545768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247daf9-8012-4185-a7aa-68f7287ad4af None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:28.556131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 683/1362] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:25:28.560546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:28.565545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:28.565545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:28.565545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:28.565545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:28.584548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc40068-aa86-4024-af1b-fd1930698a6f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:28.600554 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:28.600844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:28.601229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:28.601513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:28.601827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:28.602467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:28.602978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:28.606774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:28.630082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 680/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:25:28.631183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:28.632756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:28.671156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5f333-8279-47a0-ac11-42f79d19e72a service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:28.686972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:28.688670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:28.694310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:28.694310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] PATH_INFO: `/v3/projects/e26b6fb4664948ffbcc626f8ef0124e9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:28.694795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26b6fb4664948ffbcc626f8ef0124e9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:28.704849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:28.801956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 684/1364] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:28 2022] POST /identity/v3/projects => generated 383 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:25:28.803075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:28.807878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:28.857162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26b6fb4664948ffbcc626f8ef0124e9', 'request_id': 'req-126f3b71-1e2a-4001-8853-ceb0b5a05f83'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:28.969267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1fd7a-eaf4-4d92-a656-f94bd75387a6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:28.980438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26b6fb4664948ffbcc626f8ef0124e9', 'request_id': 'req-126f3b71-1e2a-4001-8853-ceb0b5a05f83'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:28.996510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26b6fb4664948ffbcc626f8ef0124e9', 'request_id': 'req-126f3b71-1e2a-4001-8853-ceb0b5a05f83'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.033692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] The token cache is being invalidate because project e26b6fb4664948ffbcc626f8ef0124e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:29.034115 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.035143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.037644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 681/1365] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:28 2022] DELETE /identity/v3/projects/e26b6fb4664948ffbcc626f8ef0124e9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:29.041402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:29.043037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:29.155644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:29.156343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:29.156634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:29.156924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] PATH_INFO: `/v3/projects/a2579a5bb0884e9c8a4d16d3e5dc6fcf` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:29.162609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2579a5bb0884e9c8a4d16d3e5dc6fcf)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:29.165312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:29.166983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f3b71-1e2a-4001-8853-ceb0b5a05f83 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:29.182492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:29.183602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:29.184933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:29.185448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:29.186701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:29.189561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:29.304399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 682/1366] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:29 2022] POST /identity/v3/users => generated 292 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:29.306184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:29.309777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:29.342546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bd0ca-ccb2-494d-abc0-678d16ee291f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:29.350777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2579a5bb0884e9c8a4d16d3e5dc6fcf', 'request_id': 'req-11d4ed74-d727-4851-92e5-71f22cdef197'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.362900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2579a5bb0884e9c8a4d16d3e5dc6fcf', 'request_id': 'req-11d4ed74-d727-4851-92e5-71f22cdef197'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.379886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2579a5bb0884e9c8a4d16d3e5dc6fcf', 'request_id': 'req-11d4ed74-d727-4851-92e5-71f22cdef197'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.495315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] The token cache is being invalidate because project a2579a5bb0884e9c8a4d16d3e5dc6fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:29.495674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.496353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.500991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 685/1367] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:28 2022] DELETE /identity/v3/projects/a2579a5bb0884e9c8a4d16d3e5dc6fcf => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:29.500991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:29.504251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:29.505242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:29.505772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:29.506807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] PATH_INFO: `/v3/users/26ee4461d90549ca9e44996b1f66545b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:29.508336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:29.511503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ee4461d90549ca9e44996b1f66545b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:29.514120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:29.527309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:29.541780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ed74-d727-4851-92e5-71f22cdef197 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:29.553743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:29.555922 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:29.560229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:29.560229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:29.560229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:29.562322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:29.596761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 686/1368] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:29.597722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:29.603679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:29.638179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7c395-7c76-4cec-b566-07e87aef1703 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:29.651834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:29.653198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:29.653198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:29.653390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] PATH_INFO: `/v3/users/eb817fd572aa4e3d968504c110acf1af` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:29.657829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb817fd572aa4e3d968504c110acf1af)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:29.661414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:29.728941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ee4461d90549ca9e44996b1f66545b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.729775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ee4461d90549ca9e44996b1f66545b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.750819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ee4461d90549ca9e44996b1f66545b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.759092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ee4461d90549ca9e44996b1f66545b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.760117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ee4461d90549ca9e44996b1f66545b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.881957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 683/1369] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:29 2022] DELETE /identity/v3/users/26ee4461d90549ca9e44996b1f66545b => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:29.881957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:29.884609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:29.900357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:29.937214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb817fd572aa4e3d968504c110acf1af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.956394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb817fd572aa4e3d968504c110acf1af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:29.996108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb817fd572aa4e3d968504c110acf1af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:30.041373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63102e13-023f-440d-9fb0-cfca3e0d0c25 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:30.049746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:30.050385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:30.053127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:30.053127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:30.054416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:30.059982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:30.072651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb817fd572aa4e3d968504c110acf1af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:30.073709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-294db80a-20f6-4608-a24e-8a172883ea8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb817fd572aa4e3d968504c110acf1af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:30.082546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 687/1370] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:29 2022] DELETE /identity/v3/users/eb817fd572aa4e3d968504c110acf1af => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:30.087402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c2b4-4cc9-4d3c-98b8-f92430baf5ee None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:30.090851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c2b4-4cc9-4d3c-98b8-f92430baf5ee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:30.090851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c2b4-4cc9-4d3c-98b8-f92430baf5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:30.091093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 684/1371] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:30.093878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:30.099119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:30.135287 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252383e-2340-492e-8d89-ebf53f4ab0d3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:30.203263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4d02c2b4-4cc9-4d3c-98b8-f92430baf5ee None None] MFA Rules not processed for user `495ed7e994004db99f1553ab6e0cc65c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:30.206143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4d02c2b4-4cc9-4d3c-98b8-f92430baf5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:30.290583 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8afd9-90ec-4a60-a084-97daaa2081fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:30.291907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8afd9-90ec-4a60-a084-97daaa2081fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:30.293016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8afd9-90ec-4a60-a084-97daaa2081fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:30.293570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8afd9-90ec-4a60-a084-97daaa2081fe admin admin] PATH_INFO: `/v3/projects/95fc1b55ca864735a7e9c638099b3b66/users/1bc93c2f4802443e8d8988bc9f8a01f8/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:30.634929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8afd9-90ec-4a60-a084-97daaa2081fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=95fc1b55ca864735a7e9c638099b3b66, user_id=1bc93c2f4802443e8d8988bc9f8a01f8, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:30.634929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8afd9-90ec-4a60-a084-97daaa2081fe admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:30.644528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 688/1372] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:30.644528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3d39d-40d2-438f-a7f7-2ff5f9a70df3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:30.644528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3d39d-40d2-438f-a7f7-2ff5f9a70df3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:30.644528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3d39d-40d2-438f-a7f7-2ff5f9a70df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:30.644528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dad3d39d-40d2-438f-a7f7-2ff5f9a70df3 None None] MFA Rules not processed for user `633c6c68e15f4be9a7289d15cac9de43`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:30.644528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dad3d39d-40d2-438f-a7f7-2ff5f9a70df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:30.654619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 685/1373] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:30 2022] PUT /identity/v3/projects/95fc1b55ca864735a7e9c638099b3b66/users/1bc93c2f4802443e8d8988bc9f8a01f8/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:30.654619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cffc4d-6e05-4871-88c7-63a1739c7056 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:30.654619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cffc4d-6e05-4871-88c7-63a1739c7056 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:30.654619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cffc4d-6e05-4871-88c7-63a1739c7056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:30.654619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f7cffc4d-6e05-4871-88c7-63a1739c7056 None None] MFA Rules not processed for user `495ed7e994004db99f1553ab6e0cc65c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:30.660641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f7cffc4d-6e05-4871-88c7-63a1739c7056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:30.665809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 689/1374] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:30.665809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3d39d-40d2-438f-a7f7-2ff5f9a70df3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:30.665809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dad3d39d-40d2-438f-a7f7-2ff5f9a70df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:30.665809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3d39d-40d2-438f-a7f7-2ff5f9a70df3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:30.730826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 686/1375] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:30.732700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e476ce-d034-4d28-b747-ebd516644df5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:30.733237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e476ce-d034-4d28-b747-ebd516644df5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:30.733834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e476ce-d034-4d28-b747-ebd516644df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:30.816597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-90e476ce-d034-4d28-b747-ebd516644df5 None None] MFA Rules not processed for user `e0dd5e32e4a642c8ac9e44310e0a6a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:30.818768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-90e476ce-d034-4d28-b747-ebd516644df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:30.840684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:30.841693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:30.842123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:30.842551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] PATH_INFO: `/v3/users/9939003319344ccb8cde9e9e8bab9649` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:30.844456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9939003319344ccb8cde9e9e8bab9649)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:30.847336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:30.937899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 687/1376] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:30.937899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629dd92-0484-4eb4-b8b7-e7c2600b4df5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:30.937899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629dd92-0484-4eb4-b8b7-e7c2600b4df5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:30.937899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629dd92-0484-4eb4-b8b7-e7c2600b4df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:31.017577 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a629dd92-0484-4eb4-b8b7-e7c2600b4df5 None None] MFA Rules not processed for user `1bc93c2f4802443e8d8988bc9f8a01f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:31.019161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a629dd92-0484-4eb4-b8b7-e7c2600b4df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:31.138146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 688/1377] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:31.139983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c5e6a-0d47-40d5-9935-a15cf76f6225 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:31.140377 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c5e6a-0d47-40d5-9935-a15cf76f6225 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:31.140702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c5e6a-0d47-40d5-9935-a15cf76f6225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:31.197117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3f8c5e6a-0d47-40d5-9935-a15cf76f6225 None None] MFA Rules not processed for user `633c6c68e15f4be9a7289d15cac9de43`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:31.199015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3f8c5e6a-0d47-40d5-9935-a15cf76f6225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:31.214059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9939003319344ccb8cde9e9e8bab9649'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.241827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9939003319344ccb8cde9e9e8bab9649'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.284720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9939003319344ccb8cde9e9e8bab9649'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.336696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 689/1378] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:31.342360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c5e6a-0d47-40d5-9935-a15cf76f6225 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:31.344876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3f8c5e6a-0d47-40d5-9935-a15cf76f6225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:31.369886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9939003319344ccb8cde9e9e8bab9649'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.370516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9939003319344ccb8cde9e9e8bab9649'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.382395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 690/1379] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:30 2022] DELETE /identity/v3/users/9939003319344ccb8cde9e9e8bab9649 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:31.383634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:31.386883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:31.543055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:31.578037 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9557a-1fef-48e1-ba2d-c2c2aa93d3d9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:31.584858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c5e6a-0d47-40d5-9935-a15cf76f6225 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:31.598363 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:31.599018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:31.599298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:31.599571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] PATH_INFO: `/v3/projects/c01e766de16b4abbb1867414c7ee424e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:31.602667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c01e766de16b4abbb1867414c7ee424e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:31.606925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:31.709723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:31.711825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:31.711825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:31.711825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:31.716146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:31.791607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c01e766de16b4abbb1867414c7ee424e', 'request_id': 'req-3a92a16d-2918-487b-86bb-1331d191d2a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.835076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c01e766de16b4abbb1867414c7ee424e', 'request_id': 'req-3a92a16d-2918-487b-86bb-1331d191d2a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.842364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:31.844825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:31.861207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c01e766de16b4abbb1867414c7ee424e', 'request_id': 'req-3a92a16d-2918-487b-86bb-1331d191d2a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.863545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 690/1380] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:31.864634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:31.869351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:31.890724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d39d04-1676-42f4-beff-fb155ec4ddbb service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:31.938667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] The token cache is being invalidate because project c01e766de16b4abbb1867414c7ee424e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:31.939324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.941861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:31.942706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:31.945701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 691/1381] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:31 2022] DELETE /identity/v3/projects/c01e766de16b4abbb1867414c7ee424e => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:31.947056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:31.950777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:32.164432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:32.165102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:32.165339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:32.173807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:32.188319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:32.225067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92a16d-2918-487b-86bb-1331d191d2a5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:32.231906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:32.232722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:32.233079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:32.233417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] PATH_INFO: `/v3/projects/0b54c486ea1c4ce0b28b5e5a82da7886` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:32.234755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b54c486ea1c4ce0b28b5e5a82da7886)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:32.237729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:32.380219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:32.381857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:32.415214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 691/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:32.416402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:32.419194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:32.442988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b54c486ea1c4ce0b28b5e5a82da7886', 'request_id': 'req-612c8663-013b-4547-8d31-db335808e400'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:32.473277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b54c486ea1c4ce0b28b5e5a82da7886', 'request_id': 'req-612c8663-013b-4547-8d31-db335808e400'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:32.483661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b54c486ea1c4ce0b28b5e5a82da7886', 'request_id': 'req-612c8663-013b-4547-8d31-db335808e400'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:32.493820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba07b-f039-4eb2-85e1-6d3628eff1ef service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:32.563651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] The token cache is being invalidate because project 0b54c486ea1c4ce0b28b5e5a82da7886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:32.564223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:32.566348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-612c8663-013b-4547-8d31-db335808e400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:32.572410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 692/1383] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:31 2022] DELETE /identity/v3/projects/0b54c486ea1c4ce0b28b5e5a82da7886 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:32.574365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ad786-7fda-49cd-bc08-982919ea3f3f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:32.574856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ad786-7fda-49cd-bc08-982919ea3f3f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:32.577264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ad786-7fda-49cd-bc08-982919ea3f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:32.590922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e82501-2fd2-45b8-90cc-d3fa13fda2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:32.591553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e82501-2fd2-45b8-90cc-d3fa13fda2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:32.592074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e82501-2fd2-45b8-90cc-d3fa13fda2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:32.592168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e82501-2fd2-45b8-90cc-d3fa13fda2d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:32.592863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e82501-2fd2-45b8-90cc-d3fa13fda2d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:32.595267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e82501-2fd2-45b8-90cc-d3fa13fda2d8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:32.620052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-da6ad786-7fda-49cd-bc08-982919ea3f3f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:32.626454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-da6ad786-7fda-49cd-bc08-982919ea3f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:32.652956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 693/1384] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:32.652956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7b9b7-8ce3-4469-8982-9ef1ee32ab0c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:32.652956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7b9b7-8ce3-4469-8982-9ef1ee32ab0c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:32.652956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7b9b7-8ce3-4469-8982-9ef1ee32ab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:32.702032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 692/1385] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:32 2022] POST /identity/v3/projects => generated 383 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:32.702032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf307e-2f35-4407-a32a-ddd33c20a5ff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:32.702032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf307e-2f35-4407-a32a-ddd33c20a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:32.702674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf307e-2f35-4407-a32a-ddd33c20a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:32.711840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ccc7b9b7-8ce3-4469-8982-9ef1ee32ab0c None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:32.713326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ccc7b9b7-8ce3-4469-8982-9ef1ee32ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:32.765369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f8bf307e-2f35-4407-a32a-ddd33c20a5ff None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:32.766634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f8bf307e-2f35-4407-a32a-ddd33c20a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:32.851132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 694/1386] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:32.852409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7b9b7-8ce3-4469-8982-9ef1ee32ab0c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:32.853773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ccc7b9b7-8ce3-4469-8982-9ef1ee32ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:32.884196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 693/1387] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:32.885354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf307e-2f35-4407-a32a-ddd33c20a5ff None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:32.887095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f8bf307e-2f35-4407-a32a-ddd33c20a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:32.888002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7b9b7-8ce3-4469-8982-9ef1ee32ab0c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:32.908938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffb1b6-54c6-47a4-8d45-a9bf7ac4b9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:32.909573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffb1b6-54c6-47a4-8d45-a9bf7ac4b9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:32.909852 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffb1b6-54c6-47a4-8d45-a9bf7ac4b9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:32.910124 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffb1b6-54c6-47a4-8d45-a9bf7ac4b9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:32.910762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ffb1b6-54c6-47a4-8d45-a9bf7ac4b9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:32.912740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ffb1b6-54c6-47a4-8d45-a9bf7ac4b9b3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:32.966334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:32.987257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf307e-2f35-4407-a32a-ddd33c20a5ff None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:32.995632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:32.996563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:32.996969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:32.997251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.009018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.010477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.021518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 694/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:33.022395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.024522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:33.044980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 695/1389] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:32 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:33.046263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdde57-9ddf-4498-b142-efb2d15d9b9e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.046573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdde57-9ddf-4498-b142-efb2d15d9b9e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.046820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdde57-9ddf-4498-b142-efb2d15d9b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.098634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-01fdde57-9ddf-4498-b142-efb2d15d9b9e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:33.099894 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-01fdde57-9ddf-4498-b142-efb2d15d9b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:33.120965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 696/1390] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:33.122323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565247-f5a4-49b2-8938-c0ebee8dc860 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.122605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565247-f5a4-49b2-8938-c0ebee8dc860 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.122874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565247-f5a4-49b2-8938-c0ebee8dc860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.164601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:33.166151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-05565247-f5a4-49b2-8938-c0ebee8dc860 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:33.167557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-05565247-f5a4-49b2-8938-c0ebee8dc860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:33.189797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0b79f-f000-43b7-8e63-4d9a918493e3 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.191498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 697/1391] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:33.194289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f555139-bba6-46c7-b07f-02b789749977 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.195053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f555139-bba6-46c7-b07f-02b789749977 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.196000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f555139-bba6-46c7-b07f-02b789749977 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.197932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 698/1392] 10.222.0.48 () {54 vars in 912 bytes} [Mon Oct 24 13:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:25:33.198832 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f555139-bba6-46c7-b07f-02b789749977 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.202335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b236eb-df55-4afe-8d88-a58490638712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.204796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b236eb-df55-4afe-8d88-a58490638712 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.204796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b236eb-df55-4afe-8d88-a58490638712 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.204796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b236eb-df55-4afe-8d88-a58490638712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.205594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-96b236eb-df55-4afe-8d88-a58490638712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:33.219738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f555139-bba6-46c7-b07f-02b789749977 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.225689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:33.237359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd341-9e22-4e9c-9720-1419ec0a75d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.238141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd341-9e22-4e9c-9720-1419ec0a75d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.238435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd341-9e22-4e9c-9720-1419ec0a75d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.238963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd341-9e22-4e9c-9720-1419ec0a75d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.239398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fd341-9e22-4e9c-9720-1419ec0a75d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.241481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fd341-9e22-4e9c-9720-1419ec0a75d5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.262131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 699/1393] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:33.263171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd341-9e22-4e9c-9720-1419ec0a75d5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.279054 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd341-9e22-4e9c-9720-1419ec0a75d5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.293769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.294384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.294667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.295421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.295588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:33.295899 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.297564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.311680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 700/1394] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:33.312583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.324169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b236eb-df55-4afe-8d88-a58490638712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.326076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d08484-5f24-4aac-ad07-8e3af40743f0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.326570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b236eb-df55-4afe-8d88-a58490638712 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.332012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.332705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.332990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.333722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.335292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:33.335686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.338100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.345052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 695/1395] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:33.346725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d071a-641e-4f05-9ee3-d4852c939ee5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.347239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d071a-641e-4f05-9ee3-d4852c939ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.347731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d071a-641e-4f05-9ee3-d4852c939ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.370649 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 701/1396] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:33.372133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.399375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51af7e5-b82f-4a94-bdaf-3e760dc80722 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.409078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e029f-53e8-4c16-bcab-8469c5e9a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.409909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e029f-53e8-4c16-bcab-8469c5e9a01f admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.410087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e029f-53e8-4c16-bcab-8469c5e9a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.413876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e029f-53e8-4c16-bcab-8469c5e9a01f admin admin] PATH_INFO: `/v3/projects/f1cef03d44ab4b84813e4dbea0921fb8/users/0e56ea3a07b349509769e71a27090bf7/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.422195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ac1d071a-641e-4f05-9ee3-d4852c939ee5 None None] MFA Rules not processed for user `65d4ca1062fe496dbe16d6cd1bcac1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:33.424364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ac1d071a-641e-4f05-9ee3-d4852c939ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:33.449441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e029f-53e8-4c16-bcab-8469c5e9a01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1cef03d44ab4b84813e4dbea0921fb8, user_id=0e56ea3a07b349509769e71a27090bf7, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.452956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e029f-53e8-4c16-bcab-8469c5e9a01f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.465697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 696/1397] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:33.471537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d071a-641e-4f05-9ee3-d4852c939ee5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.494348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d071a-641e-4f05-9ee3-d4852c939ee5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.501276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:33.506928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c293c-31b4-49ea-8664-da8d040dec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.509675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c293c-31b4-49ea-8664-da8d040dec08 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.509974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c293c-31b4-49ea-8664-da8d040dec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.512923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c293c-31b4-49ea-8664-da8d040dec08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.512923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c293c-31b4-49ea-8664-da8d040dec08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.516994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c293c-31b4-49ea-8664-da8d040dec08 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.536010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 702/1398] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:33 2022] PUT /identity/v3/projects/f1cef03d44ab4b84813e4dbea0921fb8/users/0e56ea3a07b349509769e71a27090bf7/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:33.536010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e029f-53e8-4c16-bcab-8469c5e9a01f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.619844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 697/1399] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:33 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:33.621165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c293c-31b4-49ea-8664-da8d040dec08 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.633924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e029f-53e8-4c16-bcab-8469c5e9a01f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.742962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c293c-31b4-49ea-8664-da8d040dec08 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.749974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4933c-51e1-4c32-9924-1ff85152c420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.751388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4933c-51e1-4c32-9924-1ff85152c420 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.751746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4933c-51e1-4c32-9924-1ff85152c420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.752651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4933c-51e1-4c32-9924-1ff85152c420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.761744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.762522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.762807 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.763084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.763407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4933c-51e1-4c32-9924-1ff85152c420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.763731 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.765540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4933c-51e1-4c32-9924-1ff85152c420 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.768058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.789580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 698/1400] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:33.790754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4933c-51e1-4c32-9924-1ff85152c420 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.829423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4933c-51e1-4c32-9924-1ff85152c420 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.844863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a9794-a201-4fec-92e8-8d0f8faa57e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.845776 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9794-a201-4fec-92e8-8d0f8faa57e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.846168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9794-a201-4fec-92e8-8d0f8faa57e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.846548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a9794-a201-4fec-92e8-8d0f8faa57e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.847417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a9794-a201-4fec-92e8-8d0f8faa57e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.849160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a9794-a201-4fec-92e8-8d0f8faa57e7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.872730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 699/1401] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:33.875124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a9794-a201-4fec-92e8-8d0f8faa57e7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.895010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a9794-a201-4fec-92e8-8d0f8faa57e7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:33.908479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed455eb-29f2-44ce-a465-552db4a12f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:33.910835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed455eb-29f2-44ce-a465-552db4a12f6e admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:33.911312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed455eb-29f2-44ce-a465-552db4a12f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:33.912223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed455eb-29f2-44ce-a465-552db4a12f6e admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:33.913247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:33.915229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed455eb-29f2-44ce-a465-552db4a12f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:33.918436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed455eb-29f2-44ce-a465-552db4a12f6e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:33.940524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 703/1402] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:33 2022] POST /identity/v3/projects => generated 365 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:33.946878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:33.957386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:33.980797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:34.020305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 700/1403] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:33 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:34.020305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed455eb-29f2-44ce-a465-552db4a12f6e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:34.045922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed455eb-29f2-44ce-a465-552db4a12f6e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:34.098207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dcb68-8f74-4e09-a07f-6ce70c82e65b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:34.106262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:34.107237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.107339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.107735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] PATH_INFO: `/v3/projects/f14cde250a534b1dab61966cb9970c2e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.113191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14cde250a534b1dab61966cb9970c2e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:34.113191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:34.174900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54871905-1917-4b90-a185-5a30d509bcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:34.175580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54871905-1917-4b90-a185-5a30d509bcc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.176333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54871905-1917-4b90-a185-5a30d509bcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.176333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54871905-1917-4b90-a185-5a30d509bcc1 admin admin] PATH_INFO: `/v3/projects/f1cef03d44ab4b84813e4dbea0921fb8/users/0e56ea3a07b349509769e71a27090bf7/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.178571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54871905-1917-4b90-a185-5a30d509bcc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1cef03d44ab4b84813e4dbea0921fb8, user_id=0e56ea3a07b349509769e71a27090bf7, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:34.182290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54871905-1917-4b90-a185-5a30d509bcc1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:34.198842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 701/1404] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:34 2022] PUT /identity/v3/projects/f1cef03d44ab4b84813e4dbea0921fb8/users/0e56ea3a07b349509769e71a27090bf7/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:34.202219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be1a2b-c611-4904-93fb-4b89f9c101e9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.210052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be1a2b-c611-4904-93fb-4b89f9c101e9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.210052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be1a2b-c611-4904-93fb-4b89f9c101e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.231031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-78be1a2b-c611-4904-93fb-4b89f9c101e9 None None] MFA Rules not processed for user `7c096064e6ec443a9e8755bc2e68dbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:34.232500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-78be1a2b-c611-4904-93fb-4b89f9c101e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:34.249970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14cde250a534b1dab61966cb9970c2e', 'request_id': 'req-aa9f0605-5581-408e-aca5-363816562aec'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:34.259638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14cde250a534b1dab61966cb9970c2e', 'request_id': 'req-aa9f0605-5581-408e-aca5-363816562aec'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:34.267092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14cde250a534b1dab61966cb9970c2e', 'request_id': 'req-aa9f0605-5581-408e-aca5-363816562aec'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:34.306367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 702/1405] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:34.306367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be1a2b-c611-4904-93fb-4b89f9c101e9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:34.354269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] The token cache is being invalidate because project f14cde250a534b1dab61966cb9970c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:34.354573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:34.356365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:34.357999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 704/1406] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:33 2022] DELETE /identity/v3/projects/f14cde250a534b1dab61966cb9970c2e => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:34.361545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:34.362063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:34.423381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:34.498786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be1a2b-c611-4904-93fb-4b89f9c101e9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:34.509424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48076c50-fbe2-4812-a5cd-4f2d44fcd597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:34.510384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076c50-fbe2-4812-a5cd-4f2d44fcd597 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.510725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076c50-fbe2-4812-a5cd-4f2d44fcd597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.511066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076c50-fbe2-4812-a5cd-4f2d44fcd597 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.511881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48076c50-fbe2-4812-a5cd-4f2d44fcd597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:34.514016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48076c50-fbe2-4812-a5cd-4f2d44fcd597 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:34.545276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f0605-5581-408e-aca5-363816562aec admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:34.564985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e63c0-715f-428c-80a4-944def26870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:34.565611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e63c0-715f-428c-80a4-944def26870e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.565893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e63c0-715f-428c-80a4-944def26870e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.566168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e63c0-715f-428c-80a4-944def26870e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.566873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e63c0-715f-428c-80a4-944def26870e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:34.568520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e63c0-715f-428c-80a4-944def26870e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:34.580734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 705/1407] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:34.587246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704d9c-8dfa-43a2-b1fd-188f61f331ba None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.587538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704d9c-8dfa-43a2-b1fd-188f61f331ba None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.587812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704d9c-8dfa-43a2-b1fd-188f61f331ba None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.589287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 706/1408] 10.222.0.48 () {54 vars in 912 bytes} [Mon Oct 24 13:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:25:34.590370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94777-372d-429d-8835-e714758f8084 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.590765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94777-372d-429d-8835-e714758f8084 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.590971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb94777-372d-429d-8835-e714758f8084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.622754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9cb94777-372d-429d-8835-e714758f8084 None None] MFA Rules not processed for user `0e56ea3a07b349509769e71a27090bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:34.623989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9cb94777-372d-429d-8835-e714758f8084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:34.638864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 703/1409] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:34 2022] POST /identity/v3/users => generated 283 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:34.643997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e00ef-9a79-4662-85b6-28fb129d050b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.644369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e00ef-9a79-4662-85b6-28fb129d050b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.645202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e00ef-9a79-4662-85b6-28fb129d050b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.676857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-857e00ef-9a79-4662-85b6-28fb129d050b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:34.678194 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-857e00ef-9a79-4662-85b6-28fb129d050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:34.687058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 704/1410] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:34.688101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e00ef-9a79-4662-85b6-28fb129d050b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:34.689519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-857e00ef-9a79-4662-85b6-28fb129d050b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:34.724703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 707/1411] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:34.727324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb94777-372d-429d-8835-e714758f8084 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:34.749764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb94777-372d-429d-8835-e714758f8084 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:34.755254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe0693-e47e-4a3d-8390-927bf65043dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:34.755871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe0693-e47e-4a3d-8390-927bf65043dd admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.756218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe0693-e47e-4a3d-8390-927bf65043dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.756565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe0693-e47e-4a3d-8390-927bf65043dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.757209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe0693-e47e-4a3d-8390-927bf65043dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:34.763100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe0693-e47e-4a3d-8390-927bf65043dd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:34.778090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 708/1412] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:34.778090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6a899-87cc-4ece-b83a-14922a6b4c0b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.778090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6a899-87cc-4ece-b83a-14922a6b4c0b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.778090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6a899-87cc-4ece-b83a-14922a6b4c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.808839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-27c6a899-87cc-4ece-b83a-14922a6b4c0b None None] MFA Rules not processed for user `65d4ca1062fe496dbe16d6cd1bcac1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:34.810818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27c6a899-87cc-4ece-b83a-14922a6b4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:34.845626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e00ef-9a79-4662-85b6-28fb129d050b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:34.870641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efb636-b3cd-4b80-a4c3-4deee19a8cbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:34.871637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efb636-b3cd-4b80-a4c3-4deee19a8cbe service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.872742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efb636-b3cd-4b80-a4c3-4deee19a8cbe service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.873254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efb636-b3cd-4b80-a4c3-4deee19a8cbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.875473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-37efb636-b3cd-4b80-a4c3-4deee19a8cbe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:34.889617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 709/1413] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:34.890487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6a899-87cc-4ece-b83a-14922a6b4c0b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:34.892920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27c6a899-87cc-4ece-b83a-14922a6b4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:34.896798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efb636-b3cd-4b80-a4c3-4deee19a8cbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:34.910682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efb636-b3cd-4b80-a4c3-4deee19a8cbe service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:34.912685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6a899-87cc-4ece-b83a-14922a6b4c0b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:34.917778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:34.918480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.918821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.919101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.919932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:34.922012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:34.934587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 705/1414] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:34.943449 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 710/1415] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:34.947465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:34.958078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48b79-6b45-4d35-b1bd-76e3b7bbec49 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:34.958078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48b79-6b45-4d35-b1bd-76e3b7bbec49 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:34.958078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48b79-6b45-4d35-b1bd-76e3b7bbec49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:34.958700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:35.012372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-13a48b79-6b45-4d35-b1bd-76e3b7bbec49 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:35.013610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-13a48b79-6b45-4d35-b1bd-76e3b7bbec49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:35.038969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 706/1416] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:35.047564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a48b79-6b45-4d35-b1bd-76e3b7bbec49 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:35.071753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a48b79-6b45-4d35-b1bd-76e3b7bbec49 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:35.099918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ed71-1629-4c2f-9075-d0ecfadc2f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:35.100620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ed71-1629-4c2f-9075-d0ecfadc2f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.100901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ed71-1629-4c2f-9075-d0ecfadc2f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.101167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ed71-1629-4c2f-9075-d0ecfadc2f43 admin admin] PATH_INFO: `/v3/projects/71018a1611e54da3839cd67986fc642b/users/30cc38fdca644704b09ff039f95f893c/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.112955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b9ae7-9553-43ce-a8f2-bdc56e6c95d3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:35.117602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:35.118474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.118832 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.119179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.127209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0ed71-1629-4c2f-9075-d0ecfadc2f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71018a1611e54da3839cd67986fc642b, user_id=30cc38fdca644704b09ff039f95f893c, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:35.159514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0ed71-1629-4c2f-9075-d0ecfadc2f43 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:35.179867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:35.182404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:35.210705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 711/1417] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:35.212847 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:35.214412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:35.237333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f195a9-d71d-4bd7-ad97-eb48b845968b service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:35.250264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839156d-4298-445e-a88b-b86714f26994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:35.251749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839156d-4298-445e-a88b-b86714f26994 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.252089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839156d-4298-445e-a88b-b86714f26994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.252797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839156d-4298-445e-a88b-b86714f26994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.270763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 707/1418] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:35 2022] PUT /identity/v3/projects/71018a1611e54da3839cd67986fc642b/users/30cc38fdca644704b09ff039f95f893c/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:35.270763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ed71-1629-4c2f-9075-d0ecfadc2f43 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:35.270914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839156d-4298-445e-a88b-b86714f26994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:35.270914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839156d-4298-445e-a88b-b86714f26994 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:35.292163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 712/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:35.299569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839156d-4298-445e-a88b-b86714f26994 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:35.307896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839156d-4298-445e-a88b-b86714f26994 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:35.399441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:35.408896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933e450-9f40-4f0b-8b8d-6250d22c9f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:35.409737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e450-9f40-4f0b-8b8d-6250d22c9f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.410854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e450-9f40-4f0b-8b8d-6250d22c9f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.410854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933e450-9f40-4f0b-8b8d-6250d22c9f55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.411293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933e450-9f40-4f0b-8b8d-6250d22c9f55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:35.411691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933e450-9f40-4f0b-8b8d-6250d22c9f55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:35.415574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933e450-9f40-4f0b-8b8d-6250d22c9f55 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:35.427207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:35.447556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 713/1420] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:35.451114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827649-be28-41a5-b327-01b58e14e5cc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.451644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827649-be28-41a5-b327-01b58e14e5cc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.452127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827649-be28-41a5-b327-01b58e14e5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.506833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ed71-1629-4c2f-9075-d0ecfadc2f43 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:35.533933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee235bba-03bf-48c0-9cbd-6a2421639959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:35.534565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee235bba-03bf-48c0-9cbd-6a2421639959 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.534840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee235bba-03bf-48c0-9cbd-6a2421639959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.535108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee235bba-03bf-48c0-9cbd-6a2421639959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.536212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee235bba-03bf-48c0-9cbd-6a2421639959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:35.546403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee235bba-03bf-48c0-9cbd-6a2421639959 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:35.564537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 708/1421] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:35.569416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee235bba-03bf-48c0-9cbd-6a2421639959 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:35.582954 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ca827649-be28-41a5-b327-01b58e14e5cc None None] MFA Rules not processed for user `30cc38fdca644704b09ff039f95f893c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:35.585583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ca827649-be28-41a5-b327-01b58e14e5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:35.685396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee235bba-03bf-48c0-9cbd-6a2421639959 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:35.695892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:35.697266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.697915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.698601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.703566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:35.767890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 714/1422] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:35.770652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca827649-be28-41a5-b327-01b58e14e5cc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:35.795196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca827649-be28-41a5-b327-01b58e14e5cc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:35.810757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185792-7e11-4135-b743-cfefa9166c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:35.811387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185792-7e11-4135-b743-cfefa9166c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.811667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185792-7e11-4135-b743-cfefa9166c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.811940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185792-7e11-4135-b743-cfefa9166c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.812618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f185792-7e11-4135-b743-cfefa9166c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:35.816548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f185792-7e11-4135-b743-cfefa9166c25 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:35.865634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:35.867302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:35.890205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 709/1423] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:35.891260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:35.904048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1a090-30f2-4550-9fbc-76045779b5ec service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:35.912567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d3ed-16b2-4f84-80f7-0368ffba6cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:35.914161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d3ed-16b2-4f84-80f7-0368ffba6cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:35.914549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d3ed-16b2-4f84-80f7-0368ffba6cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:35.915065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d3ed-16b2-4f84-80f7-0368ffba6cd2 admin admin] PATH_INFO: `/v3/projects/d5f6bfb684a14d7e8b9cf4da5f281103/users/6e34ed07023649e5bbdb9cbba54a7adf/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:35.937642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 715/1424] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:35 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:35.938559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185792-7e11-4135-b743-cfefa9166c25 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:35.942838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7d3ed-16b2-4f84-80f7-0368ffba6cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f6bfb684a14d7e8b9cf4da5f281103, user_id=6e34ed07023649e5bbdb9cbba54a7adf, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:35.946766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7d3ed-16b2-4f84-80f7-0368ffba6cd2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:36.028968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 710/1425] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:35 2022] PUT /identity/v3/projects/d5f6bfb684a14d7e8b9cf4da5f281103/users/6e34ed07023649e5bbdb9cbba54a7adf/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:36.029843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d3ed-16b2-4f84-80f7-0368ffba6cd2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:36.102595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:36.192031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185792-7e11-4135-b743-cfefa9166c25 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:36.198090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d3ed-16b2-4f84-80f7-0368ffba6cd2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:36.204014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff93c-7b0b-4cca-a896-3c245516ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:36.204780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff93c-7b0b-4cca-a896-3c245516ec7d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:36.205060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff93c-7b0b-4cca-a896-3c245516ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:36.205329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff93c-7b0b-4cca-a896-3c245516ec7d admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:36.206168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff93c-7b0b-4cca-a896-3c245516ec7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:36.218024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff93c-7b0b-4cca-a896-3c245516ec7d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:36.223097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd812da-acb9-4b8c-b4de-88e5ddabad55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:36.223729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd812da-acb9-4b8c-b4de-88e5ddabad55 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:36.224008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd812da-acb9-4b8c-b4de-88e5ddabad55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:36.224311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd812da-acb9-4b8c-b4de-88e5ddabad55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:36.236631 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd812da-acb9-4b8c-b4de-88e5ddabad55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:36.258431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd812da-acb9-4b8c-b4de-88e5ddabad55 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:36.325477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:36.347450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 711/1426] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:36 2022] POST /identity/v3/users => generated 287 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:36.347450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b1e8-748f-4846-9c9e-748e5868f938 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:36.347450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b1e8-748f-4846-9c9e-748e5868f938 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:36.347450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b1e8-748f-4846-9c9e-748e5868f938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:36.393405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a728b1e8-748f-4846-9c9e-748e5868f938 None None] MFA Rules not processed for user `6e34ed07023649e5bbdb9cbba54a7adf`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:36.394979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a728b1e8-748f-4846-9c9e-748e5868f938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:36.455575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 716/1427] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:35 2022] POST /identity/v3/projects => generated 387 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:36.455575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd812da-acb9-4b8c-b4de-88e5ddabad55 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:36.607991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 712/1428] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:36.609414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020c95c-f982-45ba-82a3-ea238abc5ba5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:36.609995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020c95c-f982-45ba-82a3-ea238abc5ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:36.610281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020c95c-f982-45ba-82a3-ea238abc5ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:36.666788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3020c95c-f982-45ba-82a3-ea238abc5ba5 None None] MFA Rules not processed for user `633c6c68e15f4be9a7289d15cac9de43`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:36.667805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd812da-acb9-4b8c-b4de-88e5ddabad55 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:36.669956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3020c95c-f982-45ba-82a3-ea238abc5ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:36.686013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd036aa-80ac-4218-b9b9-a7549a520f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:36.694074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd036aa-80ac-4218-b9b9-a7549a520f82 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:36.694074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd036aa-80ac-4218-b9b9-a7549a520f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:36.694074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd036aa-80ac-4218-b9b9-a7549a520f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:36.694074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd036aa-80ac-4218-b9b9-a7549a520f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:36.694074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd036aa-80ac-4218-b9b9-a7549a520f82 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:36.717401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 717/1429] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:36.718800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd036aa-80ac-4218-b9b9-a7549a520f82 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:36.755628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd036aa-80ac-4218-b9b9-a7549a520f82 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:36.767234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb511d84-0d6e-4acb-81ab-2c227edee0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:36.768246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb511d84-0d6e-4acb-81ab-2c227edee0ce admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:36.768568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb511d84-0d6e-4acb-81ab-2c227edee0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:36.777694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb511d84-0d6e-4acb-81ab-2c227edee0ce admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:36.777694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb511d84-0d6e-4acb-81ab-2c227edee0ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:36.777694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb511d84-0d6e-4acb-81ab-2c227edee0ce admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:36.826927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 713/1430] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:36.828011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020c95c-f982-45ba-82a3-ea238abc5ba5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:36.892872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020c95c-f982-45ba-82a3-ea238abc5ba5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:36.921656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3a24-bb70-46e9-a55c-92bba1adea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:36.924387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3a24-bb70-46e9-a55c-92bba1adea3c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:36.925317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3a24-bb70-46e9-a55c-92bba1adea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:36.927170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae3a24-bb70-46e9-a55c-92bba1adea3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:36.928843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3a24-bb70-46e9-a55c-92bba1adea3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:36.935774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae3a24-bb70-46e9-a55c-92bba1adea3c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:36.968719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 718/1431] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:36 2022] POST /identity/v3/users => generated 300 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:36.969936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb511d84-0d6e-4acb-81ab-2c227edee0ce admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.006444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb511d84-0d6e-4acb-81ab-2c227edee0ce admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.020811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb649e5-3b9a-42ce-9372-d77360d767a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:37.022158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb649e5-3b9a-42ce-9372-d77360d767a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.024417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb649e5-3b9a-42ce-9372-d77360d767a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.024691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb649e5-3b9a-42ce-9372-d77360d767a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.025354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb649e5-3b9a-42ce-9372-d77360d767a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:37.028347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb649e5-3b9a-42ce-9372-d77360d767a9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:37.048897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 719/1432] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:37.049970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb649e5-3b9a-42ce-9372-d77360d767a9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.064148 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb649e5-3b9a-42ce-9372-d77360d767a9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.074964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463b766-443d-446e-8c6d-3c1cf29b2c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:37.077180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463b766-443d-446e-8c6d-3c1cf29b2c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.077782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463b766-443d-446e-8c6d-3c1cf29b2c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.078350 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463b766-443d-446e-8c6d-3c1cf29b2c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.080018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463b766-443d-446e-8c6d-3c1cf29b2c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:37.102655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463b766-443d-446e-8c6d-3c1cf29b2c60 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:37.145527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 720/1433] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:37.148118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 714/1434] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:36 2022] POST /identity/v3/projects => generated 367 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:37.150343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463b766-443d-446e-8c6d-3c1cf29b2c60 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.152300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3a24-bb70-46e9-a55c-92bba1adea3c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.176127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463b766-443d-446e-8c6d-3c1cf29b2c60 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.296150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae3a24-bb70-46e9-a55c-92bba1adea3c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.302739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b616a-b4d3-454d-86bd-dedd1a2173c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:37.305861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b616a-b4d3-454d-86bd-dedd1a2173c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.306024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b037f9-cd36-46bd-9418-dbba052d4ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:37.306507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b616a-b4d3-454d-86bd-dedd1a2173c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.306507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b616a-b4d3-454d-86bd-dedd1a2173c8 admin admin] PATH_INFO: `/v3/projects/c461cd281ecb43149abbc1bb7258b9a4/users/42299a63c77341da8c1c8fb1e748e87e/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.306670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b037f9-cd36-46bd-9418-dbba052d4ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.306670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b037f9-cd36-46bd-9418-dbba052d4ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.306670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b037f9-cd36-46bd-9418-dbba052d4ce7 admin admin] PATH_INFO: `/v3/projects/388dfbe1d1f442df9fbd3dbdb5dd881b/users/5153dd1df5524e7b88b4c81f22106255/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.329874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b037f9-cd36-46bd-9418-dbba052d4ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388dfbe1d1f442df9fbd3dbdb5dd881b, user_id=5153dd1df5524e7b88b4c81f22106255, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:37.332977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b037f9-cd36-46bd-9418-dbba052d4ce7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:37.334883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b616a-b4d3-454d-86bd-dedd1a2173c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c461cd281ecb43149abbc1bb7258b9a4, user_id=42299a63c77341da8c1c8fb1e748e87e, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:37.337936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b616a-b4d3-454d-86bd-dedd1a2173c8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:37.434777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 721/1435] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:37 2022] PUT /identity/v3/projects/388dfbe1d1f442df9fbd3dbdb5dd881b/users/5153dd1df5524e7b88b4c81f22106255/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:37.435958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b037f9-cd36-46bd-9418-dbba052d4ce7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.501276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 715/1436] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:37 2022] PUT /identity/v3/projects/c461cd281ecb43149abbc1bb7258b9a4/users/42299a63c77341da8c1c8fb1e748e87e/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:37.502578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51708bf6-26e8-4ed6-8ac5-cbf79634a60d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.502848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51708bf6-26e8-4ed6-8ac5-cbf79634a60d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.503123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51708bf6-26e8-4ed6-8ac5-cbf79634a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.535023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b037f9-cd36-46bd-9418-dbba052d4ce7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.583991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-51708bf6-26e8-4ed6-8ac5-cbf79634a60d None None] MFA Rules not processed for user `5153dd1df5524e7b88b4c81f22106255`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:37.586729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-51708bf6-26e8-4ed6-8ac5-cbf79634a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:37.677271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e13f9-8d5a-4bbf-b963-070ad96a548d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:37.677913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e13f9-8d5a-4bbf-b963-070ad96a548d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.678190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e13f9-8d5a-4bbf-b963-070ad96a548d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.678617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e13f9-8d5a-4bbf-b963-070ad96a548d admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.680056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e13f9-8d5a-4bbf-b963-070ad96a548d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:37.684024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e13f9-8d5a-4bbf-b963-070ad96a548d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:37.720424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 716/1437] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:37.721435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51708bf6-26e8-4ed6-8ac5-cbf79634a60d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.736742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51708bf6-26e8-4ed6-8ac5-cbf79634a60d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.741909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e66bf-aed1-47bf-8056-ca68222a74e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:37.744173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e66bf-aed1-47bf-8056-ca68222a74e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.744672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e66bf-aed1-47bf-8056-ca68222a74e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.751387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e66bf-aed1-47bf-8056-ca68222a74e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.751387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e66bf-aed1-47bf-8056-ca68222a74e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:37.751387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e66bf-aed1-47bf-8056-ca68222a74e4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:37.762344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 717/1438] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:37.763374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e66bf-aed1-47bf-8056-ca68222a74e4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.773916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e66bf-aed1-47bf-8056-ca68222a74e4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.780452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43627ab-9341-4517-8355-f51bbfb63ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:37.781221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43627ab-9341-4517-8355-f51bbfb63ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.781610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43627ab-9341-4517-8355-f51bbfb63ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.782097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43627ab-9341-4517-8355-f51bbfb63ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.782892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43627ab-9341-4517-8355-f51bbfb63ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:37.788435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 722/1439] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:37 2022] POST /identity/v3/users => generated 290 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:37.788435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e13f9-8d5a-4bbf-b963-070ad96a548d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.795779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43627ab-9341-4517-8355-f51bbfb63ec3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:37.806402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e13f9-8d5a-4bbf-b963-070ad96a548d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.812625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd471b2-7a75-4d2c-9101-e8cbaf7ca12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:37.813281 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd471b2-7a75-4d2c-9101-e8cbaf7ca12b admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.813558 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd471b2-7a75-4d2c-9101-e8cbaf7ca12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.813880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd471b2-7a75-4d2c-9101-e8cbaf7ca12b admin admin] PATH_INFO: `/v3/projects/c461cd281ecb43149abbc1bb7258b9a4/users/42299a63c77341da8c1c8fb1e748e87e/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.815834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd471b2-7a75-4d2c-9101-e8cbaf7ca12b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c461cd281ecb43149abbc1bb7258b9a4, user_id=42299a63c77341da8c1c8fb1e748e87e, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:37.818697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd471b2-7a75-4d2c-9101-e8cbaf7ca12b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:37.829188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 723/1440] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:37 2022] PUT /identity/v3/projects/c461cd281ecb43149abbc1bb7258b9a4/users/42299a63c77341da8c1c8fb1e748e87e/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:37.830068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd471b2-7a75-4d2c-9101-e8cbaf7ca12b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:37.897979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 718/1441] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:37 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:37.901791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c05173-8d78-4666-b1c3-417f1261842d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:37.902077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c05173-8d78-4666-b1c3-417f1261842d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:37.902351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c05173-8d78-4666-b1c3-417f1261842d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:37.907545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd471b2-7a75-4d2c-9101-e8cbaf7ca12b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:37.937810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c5c05173-8d78-4666-b1c3-417f1261842d None None] MFA Rules not processed for user `42299a63c77341da8c1c8fb1e748e87e`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:37.939276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c5c05173-8d78-4666-b1c3-417f1261842d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:38.024107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d907-c1e1-416f-ac1d-7dee34b54af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:38.025818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d907-c1e1-416f-ac1d-7dee34b54af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.026321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d907-c1e1-416f-ac1d-7dee34b54af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.026747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d907-c1e1-416f-ac1d-7dee34b54af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.028052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d907-c1e1-416f-ac1d-7dee34b54af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:38.029941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d907-c1e1-416f-ac1d-7dee34b54af1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:38.059064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 724/1442] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:38.061364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4461-05bd-48d5-b026-384397c12e9b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.063491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4461-05bd-48d5-b026-384397c12e9b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.063805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4461-05bd-48d5-b026-384397c12e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.071770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:38.097442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f37a4461-05bd-48d5-b026-384397c12e9b None None] MFA Rules not processed for user `e0dd5e32e4a642c8ac9e44310e0a6a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:38.098687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f37a4461-05bd-48d5-b026-384397c12e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:38.115464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 719/1443] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:38.116792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c05173-8d78-4666-b1c3-417f1261842d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:38.128251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c05173-8d78-4666-b1c3-417f1261842d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:38.134136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16691542-4bf7-4f6c-89a2-23ffd40c8707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:38.135016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16691542-4bf7-4f6c-89a2-23ffd40c8707 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.135520 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16691542-4bf7-4f6c-89a2-23ffd40c8707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.135934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16691542-4bf7-4f6c-89a2-23ffd40c8707 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.136738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16691542-4bf7-4f6c-89a2-23ffd40c8707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:38.139956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16691542-4bf7-4f6c-89a2-23ffd40c8707 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:38.219751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 720/1444] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:38 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:38.219751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16691542-4bf7-4f6c-89a2-23ffd40c8707 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:38.236672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16691542-4bf7-4f6c-89a2-23ffd40c8707 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:38.237268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 725/1445] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:38.238122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4461-05bd-48d5-b026-384397c12e9b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:38.252785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd367-b05a-4310-940d-e74632749656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:38.253467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd367-b05a-4310-940d-e74632749656 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.253777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd367-b05a-4310-940d-e74632749656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.255989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd367-b05a-4310-940d-e74632749656 admin admin] PATH_INFO: `/v3/projects/3b48cb879f324ab9a758c9502c4e7268/users/f890bc2519aa4954827aaa1336dcc084/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.291832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd367-b05a-4310-940d-e74632749656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b48cb879f324ab9a758c9502c4e7268, user_id=f890bc2519aa4954827aaa1336dcc084, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:38.294779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd367-b05a-4310-940d-e74632749656 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:38.375597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4461-05bd-48d5-b026-384397c12e9b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:38.387739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 721/1446] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:38 2022] PUT /identity/v3/projects/3b48cb879f324ab9a758c9502c4e7268/users/f890bc2519aa4954827aaa1336dcc084/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:38.388742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd367-b05a-4310-940d-e74632749656 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:38.524055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649349e-fe05-4dbf-b684-c73d21f93623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:38.524867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649349e-fe05-4dbf-b684-c73d21f93623 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.525205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649349e-fe05-4dbf-b684-c73d21f93623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.525503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd367-b05a-4310-940d-e74632749656 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:38.525755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649349e-fe05-4dbf-b684-c73d21f93623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.539241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc11b2-0250-4752-83a6-f6aad0a1f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:38.539241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc11b2-0250-4752-83a6-f6aad0a1f8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.539241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc11b2-0250-4752-83a6-f6aad0a1f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.543892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc11b2-0250-4752-83a6-f6aad0a1f8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.544711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc11b2-0250-4752-83a6-f6aad0a1f8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:38.555336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc11b2-0250-4752-83a6-f6aad0a1f8f0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:38.587398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 722/1447] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:38.589097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc11b2-0250-4752-83a6-f6aad0a1f8f0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:38.618967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc11b2-0250-4752-83a6-f6aad0a1f8f0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:38.646969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08e619-f1b8-4ee4-9df4-9299c15399d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:38.647607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08e619-f1b8-4ee4-9df4-9299c15399d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.647888 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08e619-f1b8-4ee4-9df4-9299c15399d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.648190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08e619-f1b8-4ee4-9df4-9299c15399d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.648903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08e619-f1b8-4ee4-9df4-9299c15399d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:38.675351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08e619-f1b8-4ee4-9df4-9299c15399d7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:38.723685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:38.731397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649349e-fe05-4dbf-b684-c73d21f93623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:38.734100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 723/1448] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:38.737675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649349e-fe05-4dbf-b684-c73d21f93623 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:38.739513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08e619-f1b8-4ee4-9df4-9299c15399d7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:38.769754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 726/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:38.770663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649349e-fe05-4dbf-b684-c73d21f93623 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:38.782881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08e619-f1b8-4ee4-9df4-9299c15399d7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:38.799366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b469075-7846-474d-acb5-25699a3267b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:38.800049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b469075-7846-474d-acb5-25699a3267b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.803666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b469075-7846-474d-acb5-25699a3267b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.805011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b469075-7846-474d-acb5-25699a3267b7 admin admin] PATH_INFO: `/v3/projects/a68cac5285214a728bd474d1c11a8ced/users/4b85a04143fa4280861f79e42b0238cb/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.807176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649349e-fe05-4dbf-b684-c73d21f93623 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:38.824813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938de2a7-28a6-4c4e-8ced-f1dc03a7e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:38.825629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938de2a7-28a6-4c4e-8ced-f1dc03a7e20f admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.826023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938de2a7-28a6-4c4e-8ced-f1dc03a7e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.826352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938de2a7-28a6-4c4e-8ced-f1dc03a7e20f admin admin] PATH_INFO: `/v3/projects/3b48cb879f324ab9a758c9502c4e7268/users/f890bc2519aa4954827aaa1336dcc084/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.831076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938de2a7-28a6-4c4e-8ced-f1dc03a7e20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b48cb879f324ab9a758c9502c4e7268, user_id=f890bc2519aa4954827aaa1336dcc084, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:38.840198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938de2a7-28a6-4c4e-8ced-f1dc03a7e20f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:38.848185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b469075-7846-474d-acb5-25699a3267b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68cac5285214a728bd474d1c11a8ced, user_id=4b85a04143fa4280861f79e42b0238cb, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:38.851095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b469075-7846-474d-acb5-25699a3267b7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:38.856248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 727/1450] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:38 2022] PUT /identity/v3/projects/3b48cb879f324ab9a758c9502c4e7268/users/f890bc2519aa4954827aaa1336dcc084/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:38.876178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2d76c-6920-485d-8af9-55956d719097 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:38.880111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2d76c-6920-485d-8af9-55956d719097 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:38.880574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2d76c-6920-485d-8af9-55956d719097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:38.934951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b8c2d76c-6920-485d-8af9-55956d719097 None None] MFA Rules not processed for user `f890bc2519aa4954827aaa1336dcc084`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:38.936190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b8c2d76c-6920-485d-8af9-55956d719097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:38.981554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 724/1451] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:38 2022] PUT /identity/v3/projects/a68cac5285214a728bd474d1c11a8ced/users/4b85a04143fa4280861f79e42b0238cb/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:38.985429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b469075-7846-474d-acb5-25699a3267b7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:39.047570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 728/1452] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:39.065998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b469075-7846-474d-acb5-25699a3267b7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:39.069340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2d76c-6920-485d-8af9-55956d719097 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:39.072944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6cf30-333d-48a8-9588-862ee5d11eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:39.073705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6cf30-333d-48a8-9588-862ee5d11eda admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:39.074054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6cf30-333d-48a8-9588-862ee5d11eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:39.074509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6cf30-333d-48a8-9588-862ee5d11eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:39.075262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6cf30-333d-48a8-9588-862ee5d11eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:39.078758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6cf30-333d-48a8-9588-862ee5d11eda admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:39.119103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 725/1453] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:25:39.123864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6cf30-333d-48a8-9588-862ee5d11eda admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:39.137028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6cf30-333d-48a8-9588-862ee5d11eda admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:39.142942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a3b72-127b-4f5a-9ab3-bff5bc27ce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:39.144951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3b72-127b-4f5a-9ab3-bff5bc27ce70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:39.145362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3b72-127b-4f5a-9ab3-bff5bc27ce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:39.145600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a3b72-127b-4f5a-9ab3-bff5bc27ce70 admin admin] PATH_INFO: `/v3/projects/a68cac5285214a728bd474d1c11a8ced/users/4b85a04143fa4280861f79e42b0238cb/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:39.151251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a3b72-127b-4f5a-9ab3-bff5bc27ce70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68cac5285214a728bd474d1c11a8ced, user_id=4b85a04143fa4280861f79e42b0238cb, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:39.153133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a3b72-127b-4f5a-9ab3-bff5bc27ce70 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:39.166697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 726/1454] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:39 2022] PUT /identity/v3/projects/a68cac5285214a728bd474d1c11a8ced/users/4b85a04143fa4280861f79e42b0238cb/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:25:39.178631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7f1a6-5672-4a9d-b1ab-401b60e63253 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:39.179329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7f1a6-5672-4a9d-b1ab-401b60e63253 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:39.180198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7f1a6-5672-4a9d-b1ab-401b60e63253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:39.215960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-98c7f1a6-5672-4a9d-b1ab-401b60e63253 None None] MFA Rules not processed for user `4b85a04143fa4280861f79e42b0238cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:39.217679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-98c7f1a6-5672-4a9d-b1ab-401b60e63253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:39.220254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2d76c-6920-485d-8af9-55956d719097 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:39.311635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb85b2-934d-40ce-aaaa-b6540e43916a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:39.312257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb85b2-934d-40ce-aaaa-b6540e43916a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:39.312557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb85b2-934d-40ce-aaaa-b6540e43916a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:39.312828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb85b2-934d-40ce-aaaa-b6540e43916a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:39.371031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 727/1455] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:25:39.431355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb85b2-934d-40ce-aaaa-b6540e43916a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:39.433139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb85b2-934d-40ce-aaaa-b6540e43916a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:39.443424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 729/1456] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:25:39.523360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7f1a6-5672-4a9d-b1ab-401b60e63253 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:39.525261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-98c7f1a6-5672-4a9d-b1ab-401b60e63253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:39.559611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7f1a6-5672-4a9d-b1ab-401b60e63253 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:39.580460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:39.581150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:39.581487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:39.584051 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:39.592634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:39.594181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:39.615877 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 728/1457] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:39.626992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:39.807951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:39.914930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb85b2-934d-40ce-aaaa-b6540e43916a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:39.924786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb85b2-934d-40ce-aaaa-b6540e43916a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:39.931565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac044a3-3a54-4905-a91e-0583e79e2bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:39.931565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac044a3-3a54-4905-a91e-0583e79e2bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:39.931565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac044a3-3a54-4905-a91e-0583e79e2bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:39.931565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac044a3-3a54-4905-a91e-0583e79e2bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:39.991403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:39.993509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:40.028461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac044a3-3a54-4905-a91e-0583e79e2bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:40.030441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac044a3-3a54-4905-a91e-0583e79e2bf2 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:40.068244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 730/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:40.073230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac044a3-3a54-4905-a91e-0583e79e2bf2 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:40.101474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac044a3-3a54-4905-a91e-0583e79e2bf2 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:40.111146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50d23e-15c2-43f0-b0b5-60a267a16142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:40.115408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d23e-15c2-43f0-b0b5-60a267a16142 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:40.115408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d23e-15c2-43f0-b0b5-60a267a16142 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:40.115408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d23e-15c2-43f0-b0b5-60a267a16142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:40.161740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20fb5d-0b1e-4761-9224-3ac2004c383a service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:40.178956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:40.181088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:40.183305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:40.183999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] PATH_INFO: `/v3/users/1bc93c2f4802443e8d8988bc9f8a01f8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:40.187503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc93c2f4802443e8d8988bc9f8a01f8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:40.192286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:40.246253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50d23e-15c2-43f0-b0b5-60a267a16142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:40.258351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50d23e-15c2-43f0-b0b5-60a267a16142 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:40.276016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 731/1459] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:40.516718 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d5595-13a5-4f0f-af6d-62f1a509f56a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:40.517763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d5595-13a5-4f0f-af6d-62f1a509f56a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:40.520660 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d5595-13a5-4f0f-af6d-62f1a509f56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:40.626152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc93c2f4802443e8d8988bc9f8a01f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:40.629321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc93c2f4802443e8d8988bc9f8a01f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:40.662174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c28d5595-13a5-4f0f-af6d-62f1a509f56a None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:40.667662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c28d5595-13a5-4f0f-af6d-62f1a509f56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:40.679293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc93c2f4802443e8d8988bc9f8a01f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:40.700220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc93c2f4802443e8d8988bc9f8a01f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:40.701024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc93c2f4802443e8d8988bc9f8a01f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:40.822478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 732/1460] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:25:40.844132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 729/1461] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:39 2022] DELETE /identity/v3/users/1bc93c2f4802443e8d8988bc9f8a01f8 => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:40.848674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c5980-b85f-4445-b866-c4232d271809 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:40.849152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c5980-b85f-4445-b866-c4232d271809 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:40.849602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c5980-b85f-4445-b866-c4232d271809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:40.886209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5b5c5980-b85f-4445-b866-c4232d271809 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:40.889489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b5c5980-b85f-4445-b866-c4232d271809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:41.023979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:41.115719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 733/1462] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:41.127713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:41.129512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:41.352192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594a2aa-5b93-4b9e-afda-88040c59a894 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:41.354735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:41.370343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2656fa0-35c3-4403-a271-ca8ae6747661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:41.371533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2656fa0-35c3-4403-a271-ca8ae6747661 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:41.371822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2656fa0-35c3-4403-a271-ca8ae6747661 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:41.372099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2656fa0-35c3-4403-a271-ca8ae6747661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:41.419493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2656fa0-35c3-4403-a271-ca8ae6747661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:41.421437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2656fa0-35c3-4403-a271-ca8ae6747661 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:41.438793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c5980-b85f-4445-b866-c4232d271809 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:41.439491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b5c5980-b85f-4445-b866-c4232d271809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:41.464626 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 730/1463] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:41.775934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c5980-b85f-4445-b866-c4232d271809 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:41.798028 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:41.800757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:41.800757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:41.800757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:41.970650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acffd7-4100-4e03-b3a0-ff779d0dffbd None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:41.972715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:41.974463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:41.975150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acffd7-4100-4e03-b3a0-ff779d0dffbd None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:41.975680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acffd7-4100-4e03-b3a0-ff779d0dffbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:42.009879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 734/1464] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:25:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:25:42.011632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:42.017580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:42.043941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e0acffd7-4100-4e03-b3a0-ff779d0dffbd None None] MFA Rules not processed for user `65d4ca1062fe496dbe16d6cd1bcac1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:42.046150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e0acffd7-4100-4e03-b3a0-ff779d0dffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:42.096058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa318a76-0f7b-4b53-9918-61a2146ab0ea service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:42.097742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 731/1465] 10.222.0.59 () {58 vars in 1015 bytes} [Mon Oct 24 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:42.124749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:42.126859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:42.128517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:42.132018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] PATH_INFO: `/v3/users/30cc38fdca644704b09ff039f95f893c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:42.146554 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cc38fdca644704b09ff039f95f893c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:42.151934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:42.155022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acffd7-4100-4e03-b3a0-ff779d0dffbd None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:42.191007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acffd7-4100-4e03-b3a0-ff779d0dffbd None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:42.222262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4025b30-81c6-411d-abaa-84e21e3665ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:42.227726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4025b30-81c6-411d-abaa-84e21e3665ca service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:42.230011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4025b30-81c6-411d-abaa-84e21e3665ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:42.230716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4025b30-81c6-411d-abaa-84e21e3665ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:42.258231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4025b30-81c6-411d-abaa-84e21e3665ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:42.263437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4025b30-81c6-411d-abaa-84e21e3665ca service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:42.295880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 732/1466] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:42.323664 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:42.499907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cc38fdca644704b09ff039f95f893c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:42.537543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cc38fdca644704b09ff039f95f893c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:42.591875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cc38fdca644704b09ff039f95f893c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:42.603762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:42.707867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cc38fdca644704b09ff039f95f893c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:42.709229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cc38fdca644704b09ff039f95f893c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:42.717869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 735/1467] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:42 2022] DELETE /identity/v3/users/30cc38fdca644704b09ff039f95f893c => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:42.724172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d869213-f9c7-4a2a-b96b-9d7fff4ee9c2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:42.724695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d869213-f9c7-4a2a-b96b-9d7fff4ee9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:42.725072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d869213-f9c7-4a2a-b96b-9d7fff4ee9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:42.819513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6d869213-f9c7-4a2a-b96b-9d7fff4ee9c2 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:42.822727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6d869213-f9c7-4a2a-b96b-9d7fff4ee9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:42.972248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 733/1468] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:43.269825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:43.274644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:43.551729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c988c82-4f3c-4f15-aacb-2755c2b160eb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:43.560245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed11154-e2e2-4be5-a8ff-1b060635b26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:43.562966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed11154-e2e2-4be5-a8ff-1b060635b26e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:43.564657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed11154-e2e2-4be5-a8ff-1b060635b26e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:43.568879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed11154-e2e2-4be5-a8ff-1b060635b26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:43.572834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:43.587548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed11154-e2e2-4be5-a8ff-1b060635b26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:43.591769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed11154-e2e2-4be5-a8ff-1b060635b26e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:43.634282 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 736/1469] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:44.724225 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:44.821473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d869213-f9c7-4a2a-b96b-9d7fff4ee9c2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:44.823347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6d869213-f9c7-4a2a-b96b-9d7fff4ee9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:44.871864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d869213-f9c7-4a2a-b96b-9d7fff4ee9c2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:44.890434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:44.891063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:44.891350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:44.891671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] PATH_INFO: `/v3/projects/95fc1b55ca864735a7e9c638099b3b66` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:44.893043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] RBAC: Authorizing `identity:delete_project(project_id=95fc1b55ca864735a7e9c638099b3b66)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:44.899147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:45.111149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95fc1b55ca864735a7e9c638099b3b66', 'request_id': 'req-aa8a5d12-ed80-40a6-ae92-89bdd87659db'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:45.227645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95fc1b55ca864735a7e9c638099b3b66', 'request_id': 'req-aa8a5d12-ed80-40a6-ae92-89bdd87659db'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:45.264205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95fc1b55ca864735a7e9c638099b3b66', 'request_id': 'req-aa8a5d12-ed80-40a6-ae92-89bdd87659db'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:45.322450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] The token cache is being invalidate because project 95fc1b55ca864735a7e9c638099b3b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:45.322755 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:45.323698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:45.331320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 734/1470] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:44 2022] DELETE /identity/v3/projects/95fc1b55ca864735a7e9c638099b3b66 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:25:45.335495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed11154-e2e2-4be5-a8ff-1b060635b26e service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:45.368934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed11154-e2e2-4be5-a8ff-1b060635b26e service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:45.659055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:45.660047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:45.660502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:45.661012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] PATH_INFO: `/v3/users/0e56ea3a07b349509769e71a27090bf7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:45.662851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e56ea3a07b349509769e71a27090bf7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:45.666330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:45.745465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:45.931700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e56ea3a07b349509769e71a27090bf7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:45.950184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e56ea3a07b349509769e71a27090bf7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:45.989683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e56ea3a07b349509769e71a27090bf7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:46.090112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e56ea3a07b349509769e71a27090bf7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:46.090112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-9529725b-208c-4b52-9c05-35f1cf9be366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e56ea3a07b349509769e71a27090bf7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:46.114123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 737/1471] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:45 2022] DELETE /identity/v3/users/0e56ea3a07b349509769e71a27090bf7 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:46.187066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:46.438098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:46.669806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:46.671670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:46.742792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a5d12-ed80-40a6-ae92-89bdd87659db admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:46.821023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:46.988113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:46.991090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:46.991690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:46.992767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] PATH_INFO: `/v3/projects/71018a1611e54da3839cd67986fc642b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:47.002302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71018a1611e54da3839cd67986fc642b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:47.009090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:47.198326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec94cd-99cf-4446-beac-ae3b1298b8bc None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:47.199009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec94cd-99cf-4446-beac-ae3b1298b8bc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:47.199009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec94cd-99cf-4446-beac-ae3b1298b8bc None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:47.200391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 738/1472] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:25:47.243885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a52117-8b1c-4a06-a8ce-d823e33398f0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:47.255168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a52117-8b1c-4a06-a8ce-d823e33398f0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:47.255168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a52117-8b1c-4a06-a8ce-d823e33398f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:47.285093 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-14a52117-8b1c-4a06-a8ce-d823e33398f0 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:47.298041 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-14a52117-8b1c-4a06-a8ce-d823e33398f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:47.315959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71018a1611e54da3839cd67986fc642b', 'request_id': 'req-84e9e05f-faa3-48ea-b9b8-346a1474fe51'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:47.409531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71018a1611e54da3839cd67986fc642b', 'request_id': 'req-84e9e05f-faa3-48ea-b9b8-346a1474fe51'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:47.424864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71018a1611e54da3839cd67986fc642b', 'request_id': 'req-84e9e05f-faa3-48ea-b9b8-346a1474fe51'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:47.507432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] The token cache is being invalidate because project 71018a1611e54da3839cd67986fc642b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:47.508301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:47.511525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-84e9e05f-faa3-48ea-b9b8-346a1474fe51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:47.521635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 735/1473] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:46 2022] DELETE /identity/v3/projects/71018a1611e54da3839cd67986fc642b => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:47.523348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 739/1474] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:25:47.523472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10039c-4966-4c4e-b0d0-54677f56173f None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:47.523718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10039c-4966-4c4e-b0d0-54677f56173f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:47.524003 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10039c-4966-4c4e-b0d0-54677f56173f None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:47.525903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 736/1475] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:25:47.528675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a52117-8b1c-4a06-a8ce-d823e33398f0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:47.530242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-14a52117-8b1c-4a06-a8ce-d823e33398f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:47.721136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d5e5d-1741-4e7a-932f-471953e513e5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:47.722546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d5e5d-1741-4e7a-932f-471953e513e5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:47.722546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d5e5d-1741-4e7a-932f-471953e513e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:47.755832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-494d5e5d-1741-4e7a-932f-471953e513e5 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:47.757876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-494d5e5d-1741-4e7a-932f-471953e513e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:47.769363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 737/1476] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:25:47.776090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a52117-8b1c-4a06-a8ce-d823e33398f0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:47.785370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:47.786460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:47.787012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:47.787502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] PATH_INFO: `/v3/users/42299a63c77341da8c1c8fb1e748e87e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:47.790444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42299a63c77341da8c1c8fb1e748e87e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:47.792930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:47.828742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:48.049950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42299a63c77341da8c1c8fb1e748e87e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:48.079242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42299a63c77341da8c1c8fb1e748e87e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:48.117267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42299a63c77341da8c1c8fb1e748e87e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:48.192550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d5e5d-1741-4e7a-932f-471953e513e5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:48.194355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-494d5e5d-1741-4e7a-932f-471953e513e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:48.203806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42299a63c77341da8c1c8fb1e748e87e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:48.205168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42299a63c77341da8c1c8fb1e748e87e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:48.223281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d5e5d-1741-4e7a-932f-471953e513e5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:48.234351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 740/1477] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:47 2022] DELETE /identity/v3/users/42299a63c77341da8c1c8fb1e748e87e => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:48.237787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:48.240396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:48.240396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:48.240396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:48.249252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:48.287255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:48.289955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:48.309702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 738/1478] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:25:48.420517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:48.420517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:48.525290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b2799-8267-4cf2-ae07-8c81d7b43b8b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:48.531104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:48.531822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:48.532162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:48.532688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:48.534902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:48.548846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:48.551414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:48.564704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 741/1479] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:49.034481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:49.036383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:49.088963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06378-118f-4b91-9cbf-6a9b32e9b73d service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:49.184032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:49.184853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.185313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.185664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] PATH_INFO: `/v3/projects/f1cef03d44ab4b84813e4dbea0921fb8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.187513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1cef03d44ab4b84813e4dbea0921fb8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:49.189469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:49.258030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:49.261639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:49.277964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e15b8a-fdcd-4998-a217-b825861b98ee service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:49.284078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:49.285113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.285703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.286151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] PATH_INFO: `/v3/projects/c461cd281ecb43149abbc1bb7258b9a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.287712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c461cd281ecb43149abbc1bb7258b9a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:49.302583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:49.322198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:49.370505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1cef03d44ab4b84813e4dbea0921fb8', 'request_id': 'req-d63990bc-4f3e-422a-ab18-4eaa32d0c188'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.454646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1cef03d44ab4b84813e4dbea0921fb8', 'request_id': 'req-d63990bc-4f3e-422a-ab18-4eaa32d0c188'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.461399 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c461cd281ecb43149abbc1bb7258b9a4', 'request_id': 'req-13540584-57e2-46ce-95e7-befdb4c27bc6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.467834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1cef03d44ab4b84813e4dbea0921fb8', 'request_id': 'req-d63990bc-4f3e-422a-ab18-4eaa32d0c188'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.473207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c461cd281ecb43149abbc1bb7258b9a4', 'request_id': 'req-13540584-57e2-46ce-95e7-befdb4c27bc6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.483151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c461cd281ecb43149abbc1bb7258b9a4', 'request_id': 'req-13540584-57e2-46ce-95e7-befdb4c27bc6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.503705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] The token cache is being invalidate because project f1cef03d44ab4b84813e4dbea0921fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:49.504146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.504969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d63990bc-4f3e-422a-ab18-4eaa32d0c188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.506995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 739/1480] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:49 2022] DELETE /identity/v3/projects/f1cef03d44ab4b84813e4dbea0921fb8 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:49.512117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab9d3-c01d-4d8a-bd6b-d4ebb8070cd2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.512501 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab9d3-c01d-4d8a-bd6b-d4ebb8070cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.512955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfab9d3-c01d-4d8a-bd6b-d4ebb8070cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.539574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7bfab9d3-c01d-4d8a-bd6b-d4ebb8070cd2 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:49.540966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7bfab9d3-c01d-4d8a-bd6b-d4ebb8070cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:49.588132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:49.599000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] The token cache is being invalidate because project c461cd281ecb43149abbc1bb7258b9a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:49.599394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.600156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13540584-57e2-46ce-95e7-befdb4c27bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:49.605766 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 742/1481] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:49 2022] DELETE /identity/v3/projects/c461cd281ecb43149abbc1bb7258b9a4 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:49.610033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265de43c-8589-4f50-8555-950499f401a4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.610655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265de43c-8589-4f50-8555-950499f401a4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.611180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265de43c-8589-4f50-8555-950499f401a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.635597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-265de43c-8589-4f50-8555-950499f401a4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:49.637585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-265de43c-8589-4f50-8555-950499f401a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:49.771728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 743/1482] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:49.771905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 740/1483] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:49.773752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b4881-5fcf-4929-bf6c-262475657945 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.774033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b4881-5fcf-4929-bf6c-262475657945 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.774306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b4881-5fcf-4929-bf6c-262475657945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.778469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013cfa1-e425-4572-9ecd-a16f340d2109 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.778469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013cfa1-e425-4572-9ecd-a16f340d2109 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.778469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013cfa1-e425-4572-9ecd-a16f340d2109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.803588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-cb8b4881-5fcf-4929-bf6c-262475657945 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:49.805728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cb8b4881-5fcf-4929-bf6c-262475657945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:49.807063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e013cfa1-e425-4572-9ecd-a16f340d2109 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:49.808372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e013cfa1-e425-4572-9ecd-a16f340d2109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:49.818520 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 741/1484] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:49.820343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b4881-5fcf-4929-bf6c-262475657945 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:49.828697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 744/1485] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:49.829612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013cfa1-e425-4572-9ecd-a16f340d2109 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:49.831196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b4881-5fcf-4929-bf6c-262475657945 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:49.836512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:49.837256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.838308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.838308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.838722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:49.839046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:49.840076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013cfa1-e425-4572-9ecd-a16f340d2109 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:49.841178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:49.844900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:49.845498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.845795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.846062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.846755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:49.847106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:49.848738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:49.859049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 742/1486] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:49.859960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:49.868650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 745/1487] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:25:49.869599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:49.874284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe306a-d617-461c-a955-741bb576dc83 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:49.880893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27854f92-ff27-482a-8098-ef1529a1ca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:49.881735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a95978-21fe-4eee-babc-8cce87a5d57e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:49.883219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27854f92-ff27-482a-8098-ef1529a1ca15 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.883676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27854f92-ff27-482a-8098-ef1529a1ca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.884135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27854f92-ff27-482a-8098-ef1529a1ca15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.885150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27854f92-ff27-482a-8098-ef1529a1ca15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:49.887945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27854f92-ff27-482a-8098-ef1529a1ca15 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:49.888342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8ffa-a86f-4518-85e4-a1743418e600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:49.889008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8ffa-a86f-4518-85e4-a1743418e600 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:49.889287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8ffa-a86f-4518-85e4-a1743418e600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:49.895549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8ffa-a86f-4518-85e4-a1743418e600 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:49.895549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b8ffa-a86f-4518-85e4-a1743418e600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:49.895549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b8ffa-a86f-4518-85e4-a1743418e600 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:49.979407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 746/1488] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:49 2022] POST /identity/v3/projects => generated 409 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:49.980495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8ffa-a86f-4518-85e4-a1743418e600 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:49.997715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 743/1489] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:49 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:50.001875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27854f92-ff27-482a-8098-ef1529a1ca15 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.051405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8ffa-a86f-4518-85e4-a1743418e600 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.068890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27854f92-ff27-482a-8098-ef1529a1ca15 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.076947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d603e-43d2-4404-8f9f-c13319144d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.077941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d603e-43d2-4404-8f9f-c13319144d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.078519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d603e-43d2-4404-8f9f-c13319144d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.078519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d603e-43d2-4404-8f9f-c13319144d08 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.079185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d603e-43d2-4404-8f9f-c13319144d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.081809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d603e-43d2-4404-8f9f-c13319144d08 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.115056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6e2b1-c353-4d06-a717-6118d0ac56ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.115671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6e2b1-c353-4d06-a717-6118d0ac56ba admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.115950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6e2b1-c353-4d06-a717-6118d0ac56ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.116222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6e2b1-c353-4d06-a717-6118d0ac56ba admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.117451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6e2b1-c353-4d06-a717-6118d0ac56ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.119427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6e2b1-c353-4d06-a717-6118d0ac56ba admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.162766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 744/1490] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:50 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:25:50.167030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d603e-43d2-4404-8f9f-c13319144d08 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.172853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:50.176652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d603e-43d2-4404-8f9f-c13319144d08 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.181274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a594-e048-499e-8343-cd47f07b8950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.181837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a594-e048-499e-8343-cd47f07b8950 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.182118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a594-e048-499e-8343-cd47f07b8950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.182389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a594-e048-499e-8343-cd47f07b8950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.183003 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a594-e048-499e-8343-cd47f07b8950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.184676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a594-e048-499e-8343-cd47f07b8950 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.198605 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 745/1491] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:25:50.198775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 747/1492] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:49 2022] POST /identity/v3/users => generated 305 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:50.201262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a594-e048-499e-8343-cd47f07b8950 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.201473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6e2b1-c353-4d06-a717-6118d0ac56ba admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.212087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a594-e048-499e-8343-cd47f07b8950 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.212920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6e2b1-c353-4d06-a717-6118d0ac56ba admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.216746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0c76e-c8ba-499d-abb7-6f03817bb2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.217360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c76e-c8ba-499d-abb7-6f03817bb2ff admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.217461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64554c-b50a-4bef-95bc-fb441f73566d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.217632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c76e-c8ba-499d-abb7-6f03817bb2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.217903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0c76e-c8ba-499d-abb7-6f03817bb2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.217991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64554c-b50a-4bef-95bc-fb441f73566d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.218238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64554c-b50a-4bef-95bc-fb441f73566d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.218508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64554c-b50a-4bef-95bc-fb441f73566d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.218602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0c76e-c8ba-499d-abb7-6f03817bb2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.219161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64554c-b50a-4bef-95bc-fb441f73566d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.220127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0c76e-c8ba-499d-abb7-6f03817bb2ff admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.220787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64554c-b50a-4bef-95bc-fb441f73566d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.230078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 746/1493] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:50.233754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 748/1494] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:25:50.234485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64554c-b50a-4bef-95bc-fb441f73566d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.234602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0c76e-c8ba-499d-abb7-6f03817bb2ff admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.244169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0c76e-c8ba-499d-abb7-6f03817bb2ff admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.244333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64554c-b50a-4bef-95bc-fb441f73566d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.248757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c301073-42f6-45ef-b80f-dafd74e32a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.248886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170966b-1cc5-4263-aebc-eb121697778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.249788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301073-42f6-45ef-b80f-dafd74e32a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.249902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170966b-1cc5-4263-aebc-eb121697778a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.249998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301073-42f6-45ef-b80f-dafd74e32a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.250094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170966b-1cc5-4263-aebc-eb121697778a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.250176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301073-42f6-45ef-b80f-dafd74e32a0d admin admin] PATH_INFO: `/v3/projects/ec02fa1dbaa344c79caefbe2727f26af/users/6cd0df1355a846dab1eb97712f8d3abd/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.250263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170966b-1cc5-4263-aebc-eb121697778a admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.250577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170966b-1cc5-4263-aebc-eb121697778a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.252149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170966b-1cc5-4263-aebc-eb121697778a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.263607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 749/1495] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:50.265174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c301073-42f6-45ef-b80f-dafd74e32a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec02fa1dbaa344c79caefbe2727f26af, user_id=6cd0df1355a846dab1eb97712f8d3abd, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.267519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170966b-1cc5-4263-aebc-eb121697778a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.267928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c301073-42f6-45ef-b80f-dafd74e32a0d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.277690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170966b-1cc5-4263-aebc-eb121697778a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.282182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b95052-027a-4b0d-9227-711aac80695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.282809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b95052-027a-4b0d-9227-711aac80695a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.283085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b95052-027a-4b0d-9227-711aac80695a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.283355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b95052-027a-4b0d-9227-711aac80695a admin admin] PATH_INFO: `/v3/projects/1dff605f3c1f4874a1244b56a91e6853/users/3d52a70007164d09af384ed90ae4e024/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.297735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b95052-027a-4b0d-9227-711aac80695a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dff605f3c1f4874a1244b56a91e6853, user_id=3d52a70007164d09af384ed90ae4e024, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.300386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b95052-027a-4b0d-9227-711aac80695a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.317133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 747/1496] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:50 2022] PUT /identity/v3/projects/ec02fa1dbaa344c79caefbe2727f26af/users/6cd0df1355a846dab1eb97712f8d3abd/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:50.327635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ff65c-e136-4e3e-b967-8d9b31016c2c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.330364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ff65c-e136-4e3e-b967-8d9b31016c2c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.330364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ff65c-e136-4e3e-b967-8d9b31016c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.352717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 750/1497] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:50 2022] PUT /identity/v3/projects/1dff605f3c1f4874a1244b56a91e6853/users/3d52a70007164d09af384ed90ae4e024/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:25:50.357540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aa4ff65c-e136-4e3e-b967-8d9b31016c2c None None] MFA Rules not processed for user `6cd0df1355a846dab1eb97712f8d3abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:50.358733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa4ff65c-e136-4e3e-b967-8d9b31016c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:50.363677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775ab8-6494-4ea6-9d95-3319dc2033b5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.363677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775ab8-6494-4ea6-9d95-3319dc2033b5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.363677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775ab8-6494-4ea6-9d95-3319dc2033b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.402329 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ac775ab8-6494-4ea6-9d95-3319dc2033b5 None None] MFA Rules not processed for user `3d52a70007164d09af384ed90ae4e024`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:50.404677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:50.406738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ac775ab8-6494-4ea6-9d95-3319dc2033b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:50.463008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 748/1498] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:25:50.476260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ff65c-e136-4e3e-b967-8d9b31016c2c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.478109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa4ff65c-e136-4e3e-b967-8d9b31016c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:50.490496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 751/1499] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:50 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:50.495545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac775ab8-6494-4ea6-9d95-3319dc2033b5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.578735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ff65c-e136-4e3e-b967-8d9b31016c2c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.582707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8538e-2ba0-43c5-b2d5-4f62fe3fe70f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.583301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8538e-2ba0-43c5-b2d5-4f62fe3fe70f service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.583577 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8538e-2ba0-43c5-b2d5-4f62fe3fe70f service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.583847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8538e-2ba0-43c5-b2d5-4f62fe3fe70f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.584042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac775ab8-6494-4ea6-9d95-3319dc2033b5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.590497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d7d17-13d4-4a41-9ce2-a6530cf338fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.590497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d7d17-13d4-4a41-9ce2-a6530cf338fa admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.590497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d7d17-13d4-4a41-9ce2-a6530cf338fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.590497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d7d17-13d4-4a41-9ce2-a6530cf338fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.590869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d7d17-13d4-4a41-9ce2-a6530cf338fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.592597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d7d17-13d4-4a41-9ce2-a6530cf338fa admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.592942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8538e-2ba0-43c5-b2d5-4f62fe3fe70f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.594362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8538e-2ba0-43c5-b2d5-4f62fe3fe70f service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.603627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 749/1500] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:50.673490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 752/1501] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:50 2022] POST /identity/v3/projects => generated 409 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:25:50.677509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8538e-2ba0-43c5-b2d5-4f62fe3fe70f service glance] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.689207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8538e-2ba0-43c5-b2d5-4f62fe3fe70f service glance] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.747469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe83fbf-1232-4b3c-ab16-c653feb4a294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.748085 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe83fbf-1232-4b3c-ab16-c653feb4a294 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.748373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe83fbf-1232-4b3c-ab16-c653feb4a294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.748647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe83fbf-1232-4b3c-ab16-c653feb4a294 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.749269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe83fbf-1232-4b3c-ab16-c653feb4a294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.751228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe83fbf-1232-4b3c-ab16-c653feb4a294 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.785759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:50.860457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 750/1502] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:50 2022] POST /identity/v3/users => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:50.870758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d7d17-13d4-4a41-9ce2-a6530cf338fa admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.881653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d7d17-13d4-4a41-9ce2-a6530cf338fa admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.887427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee32152-6c85-4eee-b083-5a5f8de2b1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.888042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee32152-6c85-4eee-b083-5a5f8de2b1de admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.888373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee32152-6c85-4eee-b083-5a5f8de2b1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.888618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee32152-6c85-4eee-b083-5a5f8de2b1de admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.889245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee32152-6c85-4eee-b083-5a5f8de2b1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.890828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee32152-6c85-4eee-b083-5a5f8de2b1de admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:50.900968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 753/1503] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:50.916565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe83fbf-1232-4b3c-ab16-c653feb4a294 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:50.926542 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe83fbf-1232-4b3c-ab16-c653feb4a294 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:50.932301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81ba76-002f-45fb-8858-c382d04e6c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:50.932961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ba76-002f-45fb-8858-c382d04e6c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:50.933237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ba76-002f-45fb-8858-c382d04e6c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:50.933504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81ba76-002f-45fb-8858-c382d04e6c35 admin admin] PATH_INFO: `/v3/projects/251c6e9029c8479e878b88b4ebc14c49/users/05906ee99c0c442daf1e41dc42f250d5/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:50.950118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81ba76-002f-45fb-8858-c382d04e6c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251c6e9029c8479e878b88b4ebc14c49, user_id=05906ee99c0c442daf1e41dc42f250d5, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:50.952813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81ba76-002f-45fb-8858-c382d04e6c35 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:51.009211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 751/1504] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:50 2022] PUT /identity/v3/projects/251c6e9029c8479e878b88b4ebc14c49/users/05906ee99c0c442daf1e41dc42f250d5/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:25:51.013495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee32152-6c85-4eee-b083-5a5f8de2b1de admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:51.023927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee32152-6c85-4eee-b083-5a5f8de2b1de admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:51.032592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:51.095690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f4e8-a2e2-4224-8adb-e826f112bbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:51.096383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f4e8-a2e2-4224-8adb-e826f112bbfa admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:51.096743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f4e8-a2e2-4224-8adb-e826f112bbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:51.096997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f4e8-a2e2-4224-8adb-e826f112bbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:51.097555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9f4e8-a2e2-4224-8adb-e826f112bbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:51.099140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9f4e8-a2e2-4224-8adb-e826f112bbfa admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:51.110121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 754/1505] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:51.114575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81ba76-002f-45fb-8858-c382d04e6c35 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:51.125467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81ba76-002f-45fb-8858-c382d04e6c35 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:51.129990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:51.130622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:51.130914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:51.131163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] PATH_INFO: `/v3/projects/251c6e9029c8479e878b88b4ebc14c49/users/05906ee99c0c442daf1e41dc42f250d5/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:51.133234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251c6e9029c8479e878b88b4ebc14c49, user_id=05906ee99c0c442daf1e41dc42f250d5, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:51.135846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:51.145954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 752/1506] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:51 2022] PUT /identity/v3/projects/251c6e9029c8479e878b88b4ebc14c49/users/05906ee99c0c442daf1e41dc42f250d5/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:51.157605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2be1a-1d50-4590-bbb8-5a2555ba1e26 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:51.158061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2be1a-1d50-4590-bbb8-5a2555ba1e26 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:51.158343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2be1a-1d50-4590-bbb8-5a2555ba1e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:51.190202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c2b2be1a-1d50-4590-bbb8-5a2555ba1e26 None None] MFA Rules not processed for user `05906ee99c0c442daf1e41dc42f250d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:51.191411 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c2b2be1a-1d50-4590-bbb8-5a2555ba1e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:51.259873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 755/1507] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:51 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:51.270706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:51.272405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:51.341862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16b7e7-8b5c-43bd-aba4-becbb6da4806 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:51.345656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0de139-66fd-453a-b426-8c5b6288d61e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:51.346258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0de139-66fd-453a-b426-8c5b6288d61e service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:51.346537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0de139-66fd-453a-b426-8c5b6288d61e service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:51.346806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0de139-66fd-453a-b426-8c5b6288d61e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:51.356420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0de139-66fd-453a-b426-8c5b6288d61e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:51.357880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0de139-66fd-453a-b426-8c5b6288d61e service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:51.367232 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 753/1508] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:51 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:51.747934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2be1a-1d50-4590-bbb8-5a2555ba1e26 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:51.749356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c2b2be1a-1d50-4590-bbb8-5a2555ba1e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:51.761602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2be1a-1d50-4590-bbb8-5a2555ba1e26 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:51.765129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:51.765722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:51.766000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:51.766274 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:51.822882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:51.824354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:51.833345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 756/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:51 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:52.434354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3a81-113d-4483-b808-6a874e29d724 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:52.438599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3a81-113d-4483-b808-6a874e29d724 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:52.438599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a3a81-113d-4483-b808-6a874e29d724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:52.454549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-841a3a81-113d-4483-b808-6a874e29d724 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:52.463639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-841a3a81-113d-4483-b808-6a874e29d724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:52.558391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 754/1510] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:25:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:52.585970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:52.585970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:52.638217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:52.692302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d0486-c8f0-475c-bb0d-33d811d0a375 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:52.698331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023a620-52bd-477b-a22d-360ab13a89e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:52.698965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023a620-52bd-477b-a22d-360ab13a89e5 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:52.699244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023a620-52bd-477b-a22d-360ab13a89e5 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:52.699529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023a620-52bd-477b-a22d-360ab13a89e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:52.717946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023a620-52bd-477b-a22d-360ab13a89e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:52.719798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023a620-52bd-477b-a22d-360ab13a89e5 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:52.731178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 757/1511] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:25:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:25:54.463111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a3a81-113d-4483-b808-6a874e29d724 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:54.465448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-841a3a81-113d-4483-b808-6a874e29d724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:54.531456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:54.582712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a3a81-113d-4483-b808-6a874e29d724 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:54.593831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:54.594836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:54.595242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:54.596606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] PATH_INFO: `/v3/users/6e34ed07023649e5bbdb9cbba54a7adf` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:54.597913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e34ed07023649e5bbdb9cbba54a7adf)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:54.599974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:54.854370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e34ed07023649e5bbdb9cbba54a7adf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:54.855391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e34ed07023649e5bbdb9cbba54a7adf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:54.869606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e34ed07023649e5bbdb9cbba54a7adf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:54.877227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e34ed07023649e5bbdb9cbba54a7adf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:54.877983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e34ed07023649e5bbdb9cbba54a7adf'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:54.970266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 755/1512] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:54 2022] DELETE /identity/v3/users/6e34ed07023649e5bbdb9cbba54a7adf => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:54.974887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124004c-c08a-4b9b-89d3-62357dc43c65 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:54.975471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124004c-c08a-4b9b-89d3-62357dc43c65 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:54.975471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124004c-c08a-4b9b-89d3-62357dc43c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:55.018674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6124004c-c08a-4b9b-89d3-62357dc43c65 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:55.020311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6124004c-c08a-4b9b-89d3-62357dc43c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:55.129557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 758/1513] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:55.136985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:55.139901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:55.258210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da154d33-be44-4276-81b7-f5fc7894070b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:55.265672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e0676-57ba-451a-a397-119c22e990bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:55.267484 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e0676-57ba-451a-a397-119c22e990bd service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:55.268033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e0676-57ba-451a-a397-119c22e990bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:55.268533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e0676-57ba-451a-a397-119c22e990bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:55.288079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e0676-57ba-451a-a397-119c22e990bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:55.290310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e0676-57ba-451a-a397-119c22e990bd service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:55.302472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 756/1514] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:56.138102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124004c-c08a-4b9b-89d3-62357dc43c65 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:56.140230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6124004c-c08a-4b9b-89d3-62357dc43c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:56.154869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124004c-c08a-4b9b-89d3-62357dc43c65 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:56.160854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:56.161470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:56.161817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:56.162213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] PATH_INFO: `/v3/projects/d5f6bfb684a14d7e8b9cf4da5f281103` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:56.163772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f6bfb684a14d7e8b9cf4da5f281103)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:56.166536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:56.314245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f6bfb684a14d7e8b9cf4da5f281103', 'request_id': 'req-4fbaf414-5848-4eea-bccd-55f6127dd8ae'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.321383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f6bfb684a14d7e8b9cf4da5f281103', 'request_id': 'req-4fbaf414-5848-4eea-bccd-55f6127dd8ae'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.328396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f6bfb684a14d7e8b9cf4da5f281103', 'request_id': 'req-4fbaf414-5848-4eea-bccd-55f6127dd8ae'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.425132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] The token cache is being invalidate because project d5f6bfb684a14d7e8b9cf4da5f281103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:56.425509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.426135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.427643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 759/1515] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:56 2022] DELETE /identity/v3/projects/d5f6bfb684a14d7e8b9cf4da5f281103 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:56.431619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e0676-57ba-451a-a397-119c22e990bd service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:56.442099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e0676-57ba-451a-a397-119c22e990bd service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:56.523892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:56.544494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:56.545152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:56.545432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:56.545706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] PATH_INFO: `/v3/users/f890bc2519aa4954827aaa1336dcc084` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:56.546894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f890bc2519aa4954827aaa1336dcc084)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:56.548829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:56.830894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f890bc2519aa4954827aaa1336dcc084'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.840674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f890bc2519aa4954827aaa1336dcc084'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.867180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f890bc2519aa4954827aaa1336dcc084'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.888461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f890bc2519aa4954827aaa1336dcc084'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.889214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8b1bdb6e-bb5f-4dd4-b4cd-fdda170fa7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f890bc2519aa4954827aaa1336dcc084'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:56.991174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 757/1516] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:56 2022] DELETE /identity/v3/users/f890bc2519aa4954827aaa1336dcc084 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:57.489973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:57.489973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:57.501383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaf414-5848-4eea-bccd-55f6127dd8ae admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:57.554697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:57.570104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:57.570732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:57.571008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:57.571276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] PATH_INFO: `/v3/projects/3b48cb879f324ab9a758c9502c4e7268` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:57.572436 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b48cb879f324ab9a758c9502c4e7268)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:57.574336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:57.677386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b48cb879f324ab9a758c9502c4e7268', 'request_id': 'req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:57.683044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b48cb879f324ab9a758c9502c4e7268', 'request_id': 'req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:57.689285 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b48cb879f324ab9a758c9502c4e7268', 'request_id': 'req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:57.767884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] The token cache is being invalidate because project 3b48cb879f324ab9a758c9502c4e7268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:25:57.768098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:57.768712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4c9a41c-61b5-446c-bb37-e7c5b1ae18d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:25:57.770181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 760/1517] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:25:57 2022] DELETE /identity/v3/projects/3b48cb879f324ab9a758c9502c4e7268 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:57.775662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a08ba-8d4f-47b0-92c3-b6456e01521a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:57.780547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a08ba-8d4f-47b0-92c3-b6456e01521a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:57.780547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a08ba-8d4f-47b0-92c3-b6456e01521a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:57.795604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a49a08ba-8d4f-47b0-92c3-b6456e01521a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:57.796818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a49a08ba-8d4f-47b0-92c3-b6456e01521a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:57.859512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 758/1518] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:57.867790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854a3de-e335-4282-8dc6-12cea5a76219 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:57.868245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854a3de-e335-4282-8dc6-12cea5a76219 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:57.868627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854a3de-e335-4282-8dc6-12cea5a76219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:57.888364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9854a3de-e335-4282-8dc6-12cea5a76219 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:57.889543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9854a3de-e335-4282-8dc6-12cea5a76219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:57.897435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 761/1519] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:57.904546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a08ba-8d4f-47b0-92c3-b6456e01521a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:57.914707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a08ba-8d4f-47b0-92c3-b6456e01521a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:57.919417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:57.920477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:57.920758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:57.921143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:57.921889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:25:57.922217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:57.923919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:57.936044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 759/1520] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:25:57.940470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854a3de-e335-4282-8dc6-12cea5a76219 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:57.950905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854a3de-e335-4282-8dc6-12cea5a76219 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:57.957866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30b37-38af-4661-a38b-33dc90bc7941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:57.958591 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30b37-38af-4661-a38b-33dc90bc7941 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:57.958962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30b37-38af-4661-a38b-33dc90bc7941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:57.959331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30b37-38af-4661-a38b-33dc90bc7941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:57.960081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b30b37-38af-4661-a38b-33dc90bc7941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:57.962871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b30b37-38af-4661-a38b-33dc90bc7941 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:58.066836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 762/1521] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:57 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:25:58.071489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:58.081629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2b1ef-46af-4720-a336-bc60af9c6ceb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:58.145499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df627b-2997-4b7a-b64d-e5fe2adf2948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:58.146095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df627b-2997-4b7a-b64d-e5fe2adf2948 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.146371 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df627b-2997-4b7a-b64d-e5fe2adf2948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.146638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df627b-2997-4b7a-b64d-e5fe2adf2948 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.147271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df627b-2997-4b7a-b64d-e5fe2adf2948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:58.149244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df627b-2997-4b7a-b64d-e5fe2adf2948 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:58.157102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:58.266103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 760/1522] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:58 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:58.270168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30b37-38af-4661-a38b-33dc90bc7941 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:58.281215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30b37-38af-4661-a38b-33dc90bc7941 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:58.285703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c9602-384b-4787-b423-56a94822d2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:58.286371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c9602-384b-4787-b423-56a94822d2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.286719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c9602-384b-4787-b423-56a94822d2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.287118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c9602-384b-4787-b423-56a94822d2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.287826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c9602-384b-4787-b423-56a94822d2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:58.290159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c9602-384b-4787-b423-56a94822d2a9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:58.301837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 763/1523] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:58.305941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df627b-2997-4b7a-b64d-e5fe2adf2948 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:58.315644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df627b-2997-4b7a-b64d-e5fe2adf2948 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:58.320685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1ac80-93f0-4a42-9e19-f47f7b9505ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:58.321442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1ac80-93f0-4a42-9e19-f47f7b9505ed admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.322065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1ac80-93f0-4a42-9e19-f47f7b9505ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.322703 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1ac80-93f0-4a42-9e19-f47f7b9505ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.323429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1ac80-93f0-4a42-9e19-f47f7b9505ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:58.325460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1ac80-93f0-4a42-9e19-f47f7b9505ed admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:58.338138 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 761/1524] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:58.342374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c9602-384b-4787-b423-56a94822d2a9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:58.352532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c9602-384b-4787-b423-56a94822d2a9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:58.357162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd968ae-9296-4ed9-9284-be6155487242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:58.357837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd968ae-9296-4ed9-9284-be6155487242 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.358116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd968ae-9296-4ed9-9284-be6155487242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.358388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd968ae-9296-4ed9-9284-be6155487242 admin admin] PATH_INFO: `/v3/projects/e18485ec12974e698a97625273f42778/users/7c6c57b4b2db49d0b22de77812dff7df/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.376261 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd968ae-9296-4ed9-9284-be6155487242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18485ec12974e698a97625273f42778, user_id=7c6c57b4b2db49d0b22de77812dff7df, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:58.378976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd968ae-9296-4ed9-9284-be6155487242 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:58.442434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 764/1525] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:58 2022] PUT /identity/v3/projects/e18485ec12974e698a97625273f42778/users/7c6c57b4b2db49d0b22de77812dff7df/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:58.453062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574b4bf-decc-4e31-9357-ac2590b9fc3e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.453348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574b4bf-decc-4e31-9357-ac2590b9fc3e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.453623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574b4bf-decc-4e31-9357-ac2590b9fc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.490336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4574b4bf-decc-4e31-9357-ac2590b9fc3e None None] MFA Rules not processed for user `7c6c57b4b2db49d0b22de77812dff7df`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:58.496940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4574b4bf-decc-4e31-9357-ac2590b9fc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:58.562888 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 762/1526] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:58.567481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd968ae-9296-4ed9-9284-be6155487242 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:58.629930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd968ae-9296-4ed9-9284-be6155487242 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:58.635147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008674-038b-46be-bfa3-b30748a9d666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:58.635746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008674-038b-46be-bfa3-b30748a9d666 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.636021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008674-038b-46be-bfa3-b30748a9d666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.636312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008674-038b-46be-bfa3-b30748a9d666 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.636976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63008674-038b-46be-bfa3-b30748a9d666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:58.638875 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63008674-038b-46be-bfa3-b30748a9d666 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:58.654539 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:58.758486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 765/1527] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:25:58 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:58.762542 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574b4bf-decc-4e31-9357-ac2590b9fc3e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:58.773062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574b4bf-decc-4e31-9357-ac2590b9fc3e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:58.827269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700fecb-eede-4af7-b7cf-e3210a694787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:58.827912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700fecb-eede-4af7-b7cf-e3210a694787 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.828054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700fecb-eede-4af7-b7cf-e3210a694787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.828350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700fecb-eede-4af7-b7cf-e3210a694787 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.828969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700fecb-eede-4af7-b7cf-e3210a694787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:58.831030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700fecb-eede-4af7-b7cf-e3210a694787 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:58.900007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 763/1528] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:25:58 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:25:58.904026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008674-038b-46be-bfa3-b30748a9d666 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:58.914889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008674-038b-46be-bfa3-b30748a9d666 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:58.919634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a8be4-71f5-4da7-8835-c11207727ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:58.920254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a8be4-71f5-4da7-8835-c11207727ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.920556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a8be4-71f5-4da7-8835-c11207727ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.920823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a8be4-71f5-4da7-8835-c11207727ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.921489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a8be4-71f5-4da7-8835-c11207727ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:58.923094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a8be4-71f5-4da7-8835-c11207727ff3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:58.941295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 766/1529] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:25:58.945535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700fecb-eede-4af7-b7cf-e3210a694787 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:58.955475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700fecb-eede-4af7-b7cf-e3210a694787 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:58.960775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d340032-7eac-41a7-8e8a-b0bd67b3b0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:58.961415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d340032-7eac-41a7-8e8a-b0bd67b3b0da admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:58.961690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d340032-7eac-41a7-8e8a-b0bd67b3b0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:58.961959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d340032-7eac-41a7-8e8a-b0bd67b3b0da admin admin] PATH_INFO: `/v3/projects/2151f7c0abbe4ddba3732e5a946ddee5/users/4ab23f3e73264302a05f8346dda5c9c9/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:58.976807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d340032-7eac-41a7-8e8a-b0bd67b3b0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=2151f7c0abbe4ddba3732e5a946ddee5, user_id=4ab23f3e73264302a05f8346dda5c9c9, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:58.979465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d340032-7eac-41a7-8e8a-b0bd67b3b0da admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:59.056645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 764/1530] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:58 2022] PUT /identity/v3/projects/2151f7c0abbe4ddba3732e5a946ddee5/users/4ab23f3e73264302a05f8346dda5c9c9/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:25:59.057636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a8be4-71f5-4da7-8835-c11207727ff3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:59.060307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a8be4-71f5-4da7-8835-c11207727ff3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:59.123910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff40c46-fde0-4ea6-85e3-c08dac415c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:59.124635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff40c46-fde0-4ea6-85e3-c08dac415c4d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:59.125032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff40c46-fde0-4ea6-85e3-c08dac415c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:59.125374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff40c46-fde0-4ea6-85e3-c08dac415c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:59.126097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff40c46-fde0-4ea6-85e3-c08dac415c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:59.127872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff40c46-fde0-4ea6-85e3-c08dac415c4d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:59.139891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 767/1531] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:25:59.143880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d340032-7eac-41a7-8e8a-b0bd67b3b0da admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:59.170625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d340032-7eac-41a7-8e8a-b0bd67b3b0da admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:59.176697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:59.177382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:59.177760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:59.177959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] PATH_INFO: `/v3/projects/2151f7c0abbe4ddba3732e5a946ddee5/users/4ab23f3e73264302a05f8346dda5c9c9/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:59.179986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2151f7c0abbe4ddba3732e5a946ddee5, user_id=4ab23f3e73264302a05f8346dda5c9c9, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:59.182760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:59.205588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 765/1532] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:25:59 2022] PUT /identity/v3/projects/2151f7c0abbe4ddba3732e5a946ddee5/users/4ab23f3e73264302a05f8346dda5c9c9/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:25:59.210980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0ab5d-aeb8-47e6-a39f-4c4b921ffd06 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:59.210980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0ab5d-aeb8-47e6-a39f-4c4b921ffd06 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:59.210980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0ab5d-aeb8-47e6-a39f-4c4b921ffd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:59.250593 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-43a0ab5d-aeb8-47e6-a39f-4c4b921ffd06 None None] MFA Rules not processed for user `4ab23f3e73264302a05f8346dda5c9c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:59.251805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-43a0ab5d-aeb8-47e6-a39f-4c4b921ffd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:59.326029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 768/1533] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:59.337419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:59.339073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:59.366605 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:59.405814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b032889-1a74-437f-b7aa-023bc49658e9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:59.410438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6a0e6-76ad-4fcb-af66-c2ce23784ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:59.411055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a0e6-76ad-4fcb-af66-c2ce23784ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:59.411924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a0e6-76ad-4fcb-af66-c2ce23784ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:59.412441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6a0e6-76ad-4fcb-af66-c2ce23784ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:59.422619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6a0e6-76ad-4fcb-af66-c2ce23784ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:59.424044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6a0e6-76ad-4fcb-af66-c2ce23784ac3 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:59.433523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 766/1534] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:59.994507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0ab5d-aeb8-47e6-a39f-4c4b921ffd06 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:00.055479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0ab5d-aeb8-47e6-a39f-4c4b921ffd06 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:00.060501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:00.061071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:00.061350 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:00.061617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] PATH_INFO: `/v3/users/7c6c57b4b2db49d0b22de77812dff7df` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:00.062815 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6c57b4b2db49d0b22de77812dff7df)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:00.072822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:00.072822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:00.286158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6c57b4b2db49d0b22de77812dff7df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:00.295127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6c57b4b2db49d0b22de77812dff7df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:00.313006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6c57b4b2db49d0b22de77812dff7df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:00.404769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6c57b4b2db49d0b22de77812dff7df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:00.404769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6c57b4b2db49d0b22de77812dff7df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:00.433821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 769/1535] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:25:59 2022] DELETE /identity/v3/users/7c6c57b4b2db49d0b22de77812dff7df => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:00.447848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7ba93-0214-4318-bade-cfa854ab5eff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:00.448432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7ba93-0214-4318-bade-cfa854ab5eff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:00.448530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7ba93-0214-4318-bade-cfa854ab5eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:00.467900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-07d7ba93-0214-4318-bade-cfa854ab5eff None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:00.469110 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-07d7ba93-0214-4318-bade-cfa854ab5eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:00.525168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 767/1536] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:00.534640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:00.536065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:00.593950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62cb25-100b-4482-916d-3124a3ecbbe5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:00.597553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf33116-f5d9-4aa3-92a7-8d8652d738fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:00.598153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf33116-f5d9-4aa3-92a7-8d8652d738fe service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:00.598431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf33116-f5d9-4aa3-92a7-8d8652d738fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:00.598703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf33116-f5d9-4aa3-92a7-8d8652d738fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:00.608466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf33116-f5d9-4aa3-92a7-8d8652d738fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:00.609902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf33116-f5d9-4aa3-92a7-8d8652d738fe service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:00.625309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 770/1537] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:01.078906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7ba93-0214-4318-bade-cfa854ab5eff None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:01.080345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-07d7ba93-0214-4318-bade-cfa854ab5eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:01.103890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7ba93-0214-4318-bade-cfa854ab5eff None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:01.109033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:01.109664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:01.109944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:01.110215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] PATH_INFO: `/v3/projects/e18485ec12974e698a97625273f42778` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:01.111328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18485ec12974e698a97625273f42778)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:01.113718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:01.271557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18485ec12974e698a97625273f42778', 'request_id': 'req-94b4cb76-cf6c-42b6-98e3-014603118424'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.345200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18485ec12974e698a97625273f42778', 'request_id': 'req-94b4cb76-cf6c-42b6-98e3-014603118424'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.352471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18485ec12974e698a97625273f42778', 'request_id': 'req-94b4cb76-cf6c-42b6-98e3-014603118424'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.362756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] The token cache is being invalidate because project e18485ec12974e698a97625273f42778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:01.363048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.363720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.365256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 768/1538] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:01 2022] DELETE /identity/v3/projects/e18485ec12974e698a97625273f42778 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:01.370019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf33116-f5d9-4aa3-92a7-8d8652d738fe service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:01.379775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf33116-f5d9-4aa3-92a7-8d8652d738fe service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:01.437234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:01.438140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:01.438602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:01.439030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] PATH_INFO: `/v3/users/4ab23f3e73264302a05f8346dda5c9c9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:01.440652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab23f3e73264302a05f8346dda5c9c9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:01.442821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:01.631616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:01.646798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab23f3e73264302a05f8346dda5c9c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.654984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab23f3e73264302a05f8346dda5c9c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.668624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab23f3e73264302a05f8346dda5c9c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.727859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab23f3e73264302a05f8346dda5c9c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.728494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47307def-d852-4f08-9398-a14974d9cf68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab23f3e73264302a05f8346dda5c9c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:01.734883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 771/1539] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:01 2022] DELETE /identity/v3/users/4ab23f3e73264302a05f8346dda5c9c9 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:02.334820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:02.334820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:02.355140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4cb76-cf6c-42b6-98e3-014603118424 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:02.400986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:02.413449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:02.414079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:02.414361 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:02.414634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] PATH_INFO: `/v3/projects/2151f7c0abbe4ddba3732e5a946ddee5` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:02.415744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2151f7c0abbe4ddba3732e5a946ddee5)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:02.417684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:02.570202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2151f7c0abbe4ddba3732e5a946ddee5', 'request_id': 'req-7ba4c25b-c923-4889-b2ac-938eb02b1bee'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:02.630991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2151f7c0abbe4ddba3732e5a946ddee5', 'request_id': 'req-7ba4c25b-c923-4889-b2ac-938eb02b1bee'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:02.638909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2151f7c0abbe4ddba3732e5a946ddee5', 'request_id': 'req-7ba4c25b-c923-4889-b2ac-938eb02b1bee'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:02.659874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] The token cache is being invalidate because project 2151f7c0abbe4ddba3732e5a946ddee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:02.660170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:02.660864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7ba4c25b-c923-4889-b2ac-938eb02b1bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:02.664309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 769/1540] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:02 2022] DELETE /identity/v3/projects/2151f7c0abbe4ddba3732e5a946ddee5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:02.669163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff95b61-f303-4eab-a6f2-b0a8221127c3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:02.669693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff95b61-f303-4eab-a6f2-b0a8221127c3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:02.669693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff95b61-f303-4eab-a6f2-b0a8221127c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:02.709710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8ff95b61-f303-4eab-a6f2-b0a8221127c3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:02.711767 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8ff95b61-f303-4eab-a6f2-b0a8221127c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:02.788055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 772/1541] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:02.797777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ede61-022a-479d-9192-1c9a5f0716e5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:02.797777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ede61-022a-479d-9192-1c9a5f0716e5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:02.797777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ede61-022a-479d-9192-1c9a5f0716e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:02.822334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2f9ede61-022a-479d-9192-1c9a5f0716e5 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:02.824017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2f9ede61-022a-479d-9192-1c9a5f0716e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:02.834713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 770/1542] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:02.839147 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff95b61-f303-4eab-a6f2-b0a8221127c3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:02.855040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff95b61-f303-4eab-a6f2-b0a8221127c3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:02.859884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:02.860619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:02.860897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:02.861188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:02.861926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:26:02.862228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:02.863826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:02.876153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 773/1543] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:26:02.880354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ede61-022a-479d-9192-1c9a5f0716e5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:02.889264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ede61-022a-479d-9192-1c9a5f0716e5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:02.895305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aec7f3-de3a-4372-bc00-beffb6453fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:02.895950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aec7f3-de3a-4372-bc00-beffb6453fef admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:02.896416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aec7f3-de3a-4372-bc00-beffb6453fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:02.896788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aec7f3-de3a-4372-bc00-beffb6453fef admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:02.897558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aec7f3-de3a-4372-bc00-beffb6453fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:02.899461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aec7f3-de3a-4372-bc00-beffb6453fef admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:02.980771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 771/1544] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:02 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:02.984866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:02.996135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56212395-3327-4ab4-aea4-185db1012fef admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:03.055799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ba0c4-974b-402d-b1cf-40ef6e4c9138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:03.056442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ba0c4-974b-402d-b1cf-40ef6e4c9138 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:03.056720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ba0c4-974b-402d-b1cf-40ef6e4c9138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:03.056986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ba0c4-974b-402d-b1cf-40ef6e4c9138 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:03.057617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ba0c4-974b-402d-b1cf-40ef6e4c9138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:03.059516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ba0c4-974b-402d-b1cf-40ef6e4c9138 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:03.162605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 774/1545] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:02 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:03.167252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aec7f3-de3a-4372-bc00-beffb6453fef admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:03.177343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aec7f3-de3a-4372-bc00-beffb6453fef admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:03.182434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82147-796c-457d-a7fa-3386cb9f0b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:03.183024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82147-796c-457d-a7fa-3386cb9f0b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:03.183296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82147-796c-457d-a7fa-3386cb9f0b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:03.183561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb82147-796c-457d-a7fa-3386cb9f0b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:03.184216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb82147-796c-457d-a7fa-3386cb9f0b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:03.185814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb82147-796c-457d-a7fa-3386cb9f0b04 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:03.195910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 772/1546] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:03.199915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ba0c4-974b-402d-b1cf-40ef6e4c9138 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:03.210465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ba0c4-974b-402d-b1cf-40ef6e4c9138 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:03.215327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65871655-26f3-4a0e-98c1-9663bbb3a055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:03.215914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65871655-26f3-4a0e-98c1-9663bbb3a055 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:03.216196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65871655-26f3-4a0e-98c1-9663bbb3a055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:03.216498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65871655-26f3-4a0e-98c1-9663bbb3a055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:03.217185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65871655-26f3-4a0e-98c1-9663bbb3a055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:03.218767 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65871655-26f3-4a0e-98c1-9663bbb3a055 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:03.229296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 775/1547] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:03.233382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82147-796c-457d-a7fa-3386cb9f0b04 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:03.243769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb82147-796c-457d-a7fa-3386cb9f0b04 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:03.248383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:03.249023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:03.249312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:03.249565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] PATH_INFO: `/v3/projects/2e9c595d4d304afb911750e26913943e/users/6d981529a2e74d6085b644a058fd83d5/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:03.265119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9c595d4d304afb911750e26913943e, user_id=6d981529a2e74d6085b644a058fd83d5, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:03.268254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:03.375277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 773/1548] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:03 2022] PUT /identity/v3/projects/2e9c595d4d304afb911750e26913943e/users/6d981529a2e74d6085b644a058fd83d5/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:26:03.387229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e996f-2438-4b6d-8ddb-e11227adf370 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:03.387600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e996f-2438-4b6d-8ddb-e11227adf370 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:03.387947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e996f-2438-4b6d-8ddb-e11227adf370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:03.415916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:03.418935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-348e996f-2438-4b6d-8ddb-e11227adf370 None None] MFA Rules not processed for user `6d981529a2e74d6085b644a058fd83d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:03.420703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-348e996f-2438-4b6d-8ddb-e11227adf370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:03.487446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 776/1549] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:03.498709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:03.500391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:03.557210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c6e2b-a1ae-4918-98c9-b82895a91b8d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:03.564385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9de0c-e3a3-4345-84a7-2b9c34f78e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:03.564385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9de0c-e3a3-4345-84a7-2b9c34f78e0d service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:03.564385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9de0c-e3a3-4345-84a7-2b9c34f78e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:03.564385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9de0c-e3a3-4345-84a7-2b9c34f78e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:03.571738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9de0c-e3a3-4345-84a7-2b9c34f78e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:03.573234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9de0c-e3a3-4345-84a7-2b9c34f78e0d service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:03.583373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 774/1550] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:03.603283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e996f-2438-4b6d-8ddb-e11227adf370 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:03.669896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e996f-2438-4b6d-8ddb-e11227adf370 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:03.674470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:03.675097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:03.675376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:03.675647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] PATH_INFO: `/v3/users/6d981529a2e74d6085b644a058fd83d5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:03.676800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d981529a2e74d6085b644a058fd83d5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:03.679258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:03.844690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d981529a2e74d6085b644a058fd83d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:03.852391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d981529a2e74d6085b644a058fd83d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:03.865184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d981529a2e74d6085b644a058fd83d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:03.923118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d981529a2e74d6085b644a058fd83d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:03.923971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d981529a2e74d6085b644a058fd83d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:03.944176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 777/1551] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:03 2022] DELETE /identity/v3/users/6d981529a2e74d6085b644a058fd83d5 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:26:03.948635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a90e-87d1-47f8-88b5-f03c0ed5fba9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:03.948997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a90e-87d1-47f8-88b5-f03c0ed5fba9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:03.949337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a90e-87d1-47f8-88b5-f03c0ed5fba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:03.972807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c5b1a90e-87d1-47f8-88b5-f03c0ed5fba9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:03.974019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c5b1a90e-87d1-47f8-88b5-f03c0ed5fba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:03.985143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:04.047653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 775/1552] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:04.056952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:04.058547 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:04.086063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:04.118114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e7e92-d8f7-4f1a-9f9f-31baad1cf93b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:04.121690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea2f3f-162a-4e32-a768-963d49a90f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:04.122292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea2f3f-162a-4e32-a768-963d49a90f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:04.122582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea2f3f-162a-4e32-a768-963d49a90f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:04.122829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea2f3f-162a-4e32-a768-963d49a90f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:04.131568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea2f3f-162a-4e32-a768-963d49a90f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:04.133056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea2f3f-162a-4e32-a768-963d49a90f84 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:04.142256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 778/1553] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:26:04.529911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1a90e-87d1-47f8-88b5-f03c0ed5fba9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:04.534024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c5b1a90e-87d1-47f8-88b5-f03c0ed5fba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:04.554708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1a90e-87d1-47f8-88b5-f03c0ed5fba9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:04.566467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:04.567123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:04.567400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:04.567669 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] PATH_INFO: `/v3/projects/2e9c595d4d304afb911750e26913943e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:04.568854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e9c595d4d304afb911750e26913943e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:04.571447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:04.682414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e9c595d4d304afb911750e26913943e', 'request_id': 'req-1ce2152d-a414-430c-9b96-24e1990fd2f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:04.779397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e9c595d4d304afb911750e26913943e', 'request_id': 'req-1ce2152d-a414-430c-9b96-24e1990fd2f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:04.786535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e9c595d4d304afb911750e26913943e', 'request_id': 'req-1ce2152d-a414-430c-9b96-24e1990fd2f2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:04.797997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] The token cache is being invalidate because project 2e9c595d4d304afb911750e26913943e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:04.798306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:04.798916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-1ce2152d-a414-430c-9b96-24e1990fd2f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:04.800479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 776/1554] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:04 2022] DELETE /identity/v3/projects/2e9c595d4d304afb911750e26913943e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:04.805295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dd12a-7297-4af9-b0b3-24eab1538c08 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:04.805776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dd12a-7297-4af9-b0b3-24eab1538c08 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:04.805874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dd12a-7297-4af9-b0b3-24eab1538c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:04.825560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-106dd12a-7297-4af9-b0b3-24eab1538c08 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:04.826765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-106dd12a-7297-4af9-b0b3-24eab1538c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:04.887356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 779/1555] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:04.895005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f6eb3-3e5f-4d63-852c-2c7164e09280 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:04.895294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f6eb3-3e5f-4d63-852c-2c7164e09280 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:04.895569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f6eb3-3e5f-4d63-852c-2c7164e09280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:04.916286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e92f6eb3-3e5f-4d63-852c-2c7164e09280 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:04.917494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e92f6eb3-3e5f-4d63-852c-2c7164e09280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:04.925303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 777/1556] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:04.929562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dd12a-7297-4af9-b0b3-24eab1538c08 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:04.939260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dd12a-7297-4af9-b0b3-24eab1538c08 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:04.943888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:04.944529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:04.944810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:04.945089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:04.945731 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:26:04.946035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:04.947621 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:04.959317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 780/1557] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:26:04.963999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f6eb3-3e5f-4d63-852c-2c7164e09280 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:04.973197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f6eb3-3e5f-4d63-852c-2c7164e09280 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:04.978482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f27d86-1bc1-4bdb-a070-ea92602d2b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:04.979118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f27d86-1bc1-4bdb-a070-ea92602d2b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:04.979398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f27d86-1bc1-4bdb-a070-ea92602d2b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:04.979671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f27d86-1bc1-4bdb-a070-ea92602d2b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:04.980325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f27d86-1bc1-4bdb-a070-ea92602d2b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:04.982286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f27d86-1bc1-4bdb-a070-ea92602d2b88 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:05.062610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 778/1558] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:04 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:26:05.066688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:05.080590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f0077-05f8-43fe-ad1b-f36b28e20a76 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:05.160467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475603a5-aae3-46d4-bc46-f8762b9ceced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:05.161127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475603a5-aae3-46d4-bc46-f8762b9ceced admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:05.161411 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475603a5-aae3-46d4-bc46-f8762b9ceced admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:05.161686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475603a5-aae3-46d4-bc46-f8762b9ceced admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:05.162483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475603a5-aae3-46d4-bc46-f8762b9ceced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:05.164446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475603a5-aae3-46d4-bc46-f8762b9ceced admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:05.277622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 781/1559] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:05 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:26:05.281463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f27d86-1bc1-4bdb-a070-ea92602d2b88 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:05.291717 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f27d86-1bc1-4bdb-a070-ea92602d2b88 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:05.296305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239917f-4788-4e4e-9119-9db8b3ae1b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:05.296849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239917f-4788-4e4e-9119-9db8b3ae1b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:05.297129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239917f-4788-4e4e-9119-9db8b3ae1b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:05.297405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239917f-4788-4e4e-9119-9db8b3ae1b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:05.298037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239917f-4788-4e4e-9119-9db8b3ae1b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:05.299619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239917f-4788-4e4e-9119-9db8b3ae1b57 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:05.310104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 779/1560] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:05.313880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475603a5-aae3-46d4-bc46-f8762b9ceced admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:05.324936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475603a5-aae3-46d4-bc46-f8762b9ceced admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:05.329744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1aaff-2c70-4189-9e19-3cb723bf1fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:05.330426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1aaff-2c70-4189-9e19-3cb723bf1fba admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:05.330712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1aaff-2c70-4189-9e19-3cb723bf1fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:05.331382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1aaff-2c70-4189-9e19-3cb723bf1fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:05.332698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1aaff-2c70-4189-9e19-3cb723bf1fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:05.334302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1aaff-2c70-4189-9e19-3cb723bf1fba admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:05.345987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 782/1561] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:05.350220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239917f-4788-4e4e-9119-9db8b3ae1b57 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:05.362282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239917f-4788-4e4e-9119-9db8b3ae1b57 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:05.368184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:05.368893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:05.369275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:05.369627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] PATH_INFO: `/v3/projects/74bed532dc8e45de92f7c166baa43374/users/f9c8502625604b2ea45808763f39e7ee/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:05.386366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bed532dc8e45de92f7c166baa43374, user_id=f9c8502625604b2ea45808763f39e7ee, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:05.389141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:05.460965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 780/1562] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:05 2022] PUT /identity/v3/projects/74bed532dc8e45de92f7c166baa43374/users/f9c8502625604b2ea45808763f39e7ee/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:05.471414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a57c9-060f-4ed5-98e4-54f59a57678f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:05.471701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a57c9-060f-4ed5-98e4-54f59a57678f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:05.471976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a57c9-060f-4ed5-98e4-54f59a57678f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:05.500394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-200a57c9-060f-4ed5-98e4-54f59a57678f None None] MFA Rules not processed for user `f9c8502625604b2ea45808763f39e7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:05.501681 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-200a57c9-060f-4ed5-98e4-54f59a57678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:05.520742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:05.572344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 783/1563] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:05.583331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:05.585026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:05.627007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:05.642475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c18fa-144d-4d34-ba64-a4298b672dc6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:05.646135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d05226-7dcf-4b57-9826-cac795601590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:05.646743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d05226-7dcf-4b57-9826-cac795601590 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:05.647022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d05226-7dcf-4b57-9826-cac795601590 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:05.647309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d05226-7dcf-4b57-9826-cac795601590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:05.655857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d05226-7dcf-4b57-9826-cac795601590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:05.657331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d05226-7dcf-4b57-9826-cac795601590 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:05.666271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 781/1564] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:15.653831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=18 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:15.769535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:25.401797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a57c9-060f-4ed5-98e4-54f59a57678f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:25.405390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-200a57c9-060f-4ed5-98e4-54f59a57678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:25.548361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a57c9-060f-4ed5-98e4-54f59a57678f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:25.554257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:25.554914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:25.555197 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:25.555474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] PATH_INFO: `/v3/users/3d52a70007164d09af384ed90ae4e024` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:25.556856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d52a70007164d09af384ed90ae4e024)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:25.559931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:25.756212 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d52a70007164d09af384ed90ae4e024'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:25.793184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d52a70007164d09af384ed90ae4e024'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:25.837435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d52a70007164d09af384ed90ae4e024'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:25.910650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d52a70007164d09af384ed90ae4e024'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:25.911330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d52a70007164d09af384ed90ae4e024'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:25.919369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 784/1565] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:25 2022] DELETE /identity/v3/users/3d52a70007164d09af384ed90ae4e024 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:25.925545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78623-2c08-4815-88cc-52cf8672bdca None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:25.926074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78623-2c08-4815-88cc-52cf8672bdca None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:25.926167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78623-2c08-4815-88cc-52cf8672bdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:25.951753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-60b78623-2c08-4815-88cc-52cf8672bdca None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:25.953204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-60b78623-2c08-4815-88cc-52cf8672bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:26.084400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 782/1566] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:26.104065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:26.105569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:26.196578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f67a59-4128-4e7f-b497-a20f3b518ce2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:26.200109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8bb98-01f8-4dd9-9291-a16cd840122e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:26.201106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8bb98-01f8-4dd9-9291-a16cd840122e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:26.201452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8bb98-01f8-4dd9-9291-a16cd840122e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:26.201701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8bb98-01f8-4dd9-9291-a16cd840122e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:26.211835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8bb98-01f8-4dd9-9291-a16cd840122e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:26.213270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8bb98-01f8-4dd9-9291-a16cd840122e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:26.222841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 785/1567] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:26.666092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78623-2c08-4815-88cc-52cf8672bdca None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:26.667591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-60b78623-2c08-4815-88cc-52cf8672bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:26.681630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78623-2c08-4815-88cc-52cf8672bdca None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:26.688203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:26.688474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:26.688764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:26.689026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] PATH_INFO: `/v3/projects/1dff605f3c1f4874a1244b56a91e6853` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:26.690236 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dff605f3c1f4874a1244b56a91e6853)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:26.692456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:26.798869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dff605f3c1f4874a1244b56a91e6853', 'request_id': 'req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:26.864648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dff605f3c1f4874a1244b56a91e6853', 'request_id': 'req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:26.871987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dff605f3c1f4874a1244b56a91e6853', 'request_id': 'req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:26.884397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] The token cache is being invalidate because project 1dff605f3c1f4874a1244b56a91e6853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:26.885219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:26.885676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b201bf4c-0e80-40c2-89d0-7d67ed18f61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:26.887253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 783/1568] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:26 2022] DELETE /identity/v3/projects/1dff605f3c1f4874a1244b56a91e6853 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:26.891650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8bb98-01f8-4dd9-9291-a16cd840122e service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:26.902142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8bb98-01f8-4dd9-9291-a16cd840122e service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:26.921038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:26.964486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:26.965137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:26.965408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:26.965672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] PATH_INFO: `/v3/users/05906ee99c0c442daf1e41dc42f250d5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:26.966833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05906ee99c0c442daf1e41dc42f250d5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:26.968738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:27.212634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05906ee99c0c442daf1e41dc42f250d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:27.224083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05906ee99c0c442daf1e41dc42f250d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:27.239333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05906ee99c0c442daf1e41dc42f250d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:27.300978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd5c34-738d-476e-bdb3-e68a2e030e48 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:27.301526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd5c34-738d-476e-bdb3-e68a2e030e48 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:27.301526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd5c34-738d-476e-bdb3-e68a2e030e48 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:27.302938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 784/1569] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:26:27.326932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84484e-7f8a-4048-9490-bd49048e7c26 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:27.327608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84484e-7f8a-4048-9490-bd49048e7c26 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:27.327912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84484e-7f8a-4048-9490-bd49048e7c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:27.339994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05906ee99c0c442daf1e41dc42f250d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:27.342259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05906ee99c0c442daf1e41dc42f250d5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:27.349567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-de84484e-7f8a-4048-9490-bd49048e7c26 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:27.357546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 786/1570] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:26 2022] DELETE /identity/v3/users/05906ee99c0c442daf1e41dc42f250d5 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:27.357680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-de84484e-7f8a-4048-9490-bd49048e7c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:27.441952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 785/1571] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:27.517635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:27.518410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:27.535189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c079f-f0f4-4e8a-92b6-169d8325d4f0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:27.541506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8ec3-817d-4d98-8bef-9e082a630ef0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:27.542201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ec3-817d-4d98-8bef-9e082a630ef0 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:27.542572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ec3-817d-4d98-8bef-9e082a630ef0 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:27.543127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ec3-817d-4d98-8bef-9e082a630ef0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:27.553369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8ec3-817d-4d98-8bef-9e082a630ef0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:27.556446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b8ec3-817d-4d98-8bef-9e082a630ef0 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:27.586467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 787/1572] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:27.843324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84484e-7f8a-4048-9490-bd49048e7c26 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:27.847341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-de84484e-7f8a-4048-9490-bd49048e7c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:27.867633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:27.917019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84484e-7f8a-4048-9490-bd49048e7c26 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:27.922982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:27.923708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:27.924074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:27.924618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] PATH_INFO: `/v3/projects/251c6e9029c8479e878b88b4ebc14c49` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:27.925987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=251c6e9029c8479e878b88b4ebc14c49)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:27.928123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:28.073335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251c6e9029c8479e878b88b4ebc14c49', 'request_id': 'req-076ba5bd-bf34-49a9-afb2-33301bb831eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:28.144370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251c6e9029c8479e878b88b4ebc14c49', 'request_id': 'req-076ba5bd-bf34-49a9-afb2-33301bb831eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:28.151759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251c6e9029c8479e878b88b4ebc14c49', 'request_id': 'req-076ba5bd-bf34-49a9-afb2-33301bb831eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:28.162333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] The token cache is being invalidate because project 251c6e9029c8479e878b88b4ebc14c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:28.162622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:28.163276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-076ba5bd-bf34-49a9-afb2-33301bb831eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:28.165416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 786/1573] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:27 2022] DELETE /identity/v3/projects/251c6e9029c8479e878b88b4ebc14c49 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:28.170506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d47f-a1c7-4c7d-8801-b8b1f1996c87 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.170972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d47f-a1c7-4c7d-8801-b8b1f1996c87 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.171103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3d47f-a1c7-4c7d-8801-b8b1f1996c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.190856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0ce3d47f-a1c7-4c7d-8801-b8b1f1996c87 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:28.192205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0ce3d47f-a1c7-4c7d-8801-b8b1f1996c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:28.254573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 788/1574] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:28.259842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e392ddc-b1d1-4ccb-a279-3319c2edcf3a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.260126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e392ddc-b1d1-4ccb-a279-3319c2edcf3a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.261127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e392ddc-b1d1-4ccb-a279-3319c2edcf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.280416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1e392ddc-b1d1-4ccb-a279-3319c2edcf3a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:28.281603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1e392ddc-b1d1-4ccb-a279-3319c2edcf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:28.290607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 787/1575] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:28.294696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3d47f-a1c7-4c7d-8801-b8b1f1996c87 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:28.304174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3d47f-a1c7-4c7d-8801-b8b1f1996c87 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:28.308721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:28.309314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.309588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.309854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.310495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:26:28.310787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:28.312390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:28.327618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 789/1576] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:26:28.332581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e392ddc-b1d1-4ccb-a279-3319c2edcf3a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:28.343339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e392ddc-b1d1-4ccb-a279-3319c2edcf3a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:28.353494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:28.354621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.355113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.355528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.356861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:28.358960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:28.494343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 788/1577] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:28 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:28.494803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:28.521104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751aea1-2a01-453e-88d8-6e314d16b331 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:28.537828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:28.566107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:28.568342 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:28.597589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe85a53-b546-4661-a201-5008ea316bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:28.601422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85a53-b546-4661-a201-5008ea316bab admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.601968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85a53-b546-4661-a201-5008ea316bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.606746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe85a53-b546-4661-a201-5008ea316bab admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.606746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe85a53-b546-4661-a201-5008ea316bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:28.613129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe85a53-b546-4661-a201-5008ea316bab admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:28.717121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdd1fb-76db-4fed-86e1-68d924982979 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:28.737295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:28.737506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.737791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.738071 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] PATH_INFO: `/v3/users/f9c8502625604b2ea45808763f39e7ee` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.739310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c8502625604b2ea45808763f39e7ee)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:28.741306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:28.763111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 790/1578] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:28 2022] POST /identity/v3/users => generated 292 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:28.768048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe85a53-b546-4661-a201-5008ea316bab admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:28.773160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:28.782932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe85a53-b546-4661-a201-5008ea316bab admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:28.789923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e583c44-7c5b-491f-8d06-8469afb1678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:28.790544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e583c44-7c5b-491f-8d06-8469afb1678b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.790827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e583c44-7c5b-491f-8d06-8469afb1678b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.791318 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e583c44-7c5b-491f-8d06-8469afb1678b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.791970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e583c44-7c5b-491f-8d06-8469afb1678b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:28.793642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e583c44-7c5b-491f-8d06-8469afb1678b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:28.804723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 791/1579] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:26:28.808666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e583c44-7c5b-491f-8d06-8469afb1678b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:28.818648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e583c44-7c5b-491f-8d06-8469afb1678b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:28.828422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56dd6e-c9cb-40c0-9f63-6421a931b4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:28.829227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56dd6e-c9cb-40c0-9f63-6421a931b4df admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.829833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56dd6e-c9cb-40c0-9f63-6421a931b4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.830203 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56dd6e-c9cb-40c0-9f63-6421a931b4df admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.832011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56dd6e-c9cb-40c0-9f63-6421a931b4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:28.834206 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56dd6e-c9cb-40c0-9f63-6421a931b4df admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:28.851999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 792/1580] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:26:28.857433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56dd6e-c9cb-40c0-9f63-6421a931b4df admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:28.869932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56dd6e-c9cb-40c0-9f63-6421a931b4df admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:28.875715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66b7ee-876c-491c-9345-37d1f8b6bc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:28.876218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b7ee-876c-491c-9345-37d1f8b6bc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.876587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b7ee-876c-491c-9345-37d1f8b6bc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.876968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b7ee-876c-491c-9345-37d1f8b6bc47 admin admin] PATH_INFO: `/v3/projects/391f4d97725b4f8280c43f4b32b3e4c1/users/81108c59dc434b459477edc7893b6fd4/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:28.896522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66b7ee-876c-491c-9345-37d1f8b6bc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391f4d97725b4f8280c43f4b32b3e4c1, user_id=81108c59dc434b459477edc7893b6fd4, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:28.899455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66b7ee-876c-491c-9345-37d1f8b6bc47 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:28.940818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c8502625604b2ea45808763f39e7ee'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:28.943215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c8502625604b2ea45808763f39e7ee'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:28.980995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 793/1581] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:28 2022] PUT /identity/v3/projects/391f4d97725b4f8280c43f4b32b3e4c1/users/81108c59dc434b459477edc7893b6fd4/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:28.993913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131913ef-03c6-4fdc-866f-2e01f29ea5f1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:28.994372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131913ef-03c6-4fdc-866f-2e01f29ea5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:28.994749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131913ef-03c6-4fdc-866f-2e01f29ea5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:29.014968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c8502625604b2ea45808763f39e7ee'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:29.024014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c8502625604b2ea45808763f39e7ee'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:29.024822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-87788ff3-fc43-4247-b3e2-269d1f5c2b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c8502625604b2ea45808763f39e7ee'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:29.034537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-131913ef-03c6-4fdc-866f-2e01f29ea5f1 None None] MFA Rules not processed for user `81108c59dc434b459477edc7893b6fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:29.036059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-131913ef-03c6-4fdc-866f-2e01f29ea5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:29.128969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 789/1582] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:28 2022] DELETE /identity/v3/users/f9c8502625604b2ea45808763f39e7ee => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:29.133510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e162c1-8d55-4b31-ac82-4b857a15a4bc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:29.134262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e162c1-8d55-4b31-ac82-4b857a15a4bc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:29.134262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e162c1-8d55-4b31-ac82-4b857a15a4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:29.160603 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 794/1583] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:29.161616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131913ef-03c6-4fdc-866f-2e01f29ea5f1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:29.167884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-131913ef-03c6-4fdc-866f-2e01f29ea5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:29.186586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-95e162c1-8d55-4b31-ac82-4b857a15a4bc None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:29.187860 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-95e162c1-8d55-4b31-ac82-4b857a15a4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:29.285063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:29.307137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 790/1584] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:29.312372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131913ef-03c6-4fdc-866f-2e01f29ea5f1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:29.324885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c297d-e7da-4e7b-9daa-2fabea5f1565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:29.326259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c297d-e7da-4e7b-9daa-2fabea5f1565 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:29.326606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c297d-e7da-4e7b-9daa-2fabea5f1565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:29.326946 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c297d-e7da-4e7b-9daa-2fabea5f1565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:29.328339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c297d-e7da-4e7b-9daa-2fabea5f1565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:29.334237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c297d-e7da-4e7b-9daa-2fabea5f1565 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:29.347721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e162c1-8d55-4b31-ac82-4b857a15a4bc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:29.349268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-95e162c1-8d55-4b31-ac82-4b857a15a4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:29.473487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 795/1585] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:29 2022] POST /identity/v3/projects => generated 381 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:29.480337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e162c1-8d55-4b31-ac82-4b857a15a4bc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:29.495077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c297d-e7da-4e7b-9daa-2fabea5f1565 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:29.602900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c086d-1748-480e-84a2-08e2e0548208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:29.609940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c086d-1748-480e-84a2-08e2e0548208 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:29.609940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c086d-1748-480e-84a2-08e2e0548208 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:29.609940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c086d-1748-480e-84a2-08e2e0548208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:29.618380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:29.646581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c297d-e7da-4e7b-9daa-2fabea5f1565 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:29.661695 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a0cad-dd0d-4384-9bed-ad67644ae2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:29.662629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a0cad-dd0d-4384-9bed-ad67644ae2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:29.669037 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a0cad-dd0d-4384-9bed-ad67644ae2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:29.669451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a0cad-dd0d-4384-9bed-ad67644ae2e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:29.670282 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a0cad-dd0d-4384-9bed-ad67644ae2e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:29.674524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a0cad-dd0d-4384-9bed-ad67644ae2e2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:29.733992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c086d-1748-480e-84a2-08e2e0548208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:29.737948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c086d-1748-480e-84a2-08e2e0548208 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:29.751206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 791/1586] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:26:29.769231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c086d-1748-480e-84a2-08e2e0548208 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:29.769719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 796/1587] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:29 2022] POST /identity/v3/users => generated 297 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:29.781376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c086d-1748-480e-84a2-08e2e0548208 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:29.812350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383ffd7-638f-4940-920e-24c8ce1c15a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:29.813534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383ffd7-638f-4940-920e-24c8ce1c15a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:29.813534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383ffd7-638f-4940-920e-24c8ce1c15a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:29.813903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383ffd7-638f-4940-920e-24c8ce1c15a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:29.814652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383ffd7-638f-4940-920e-24c8ce1c15a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:29.816582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383ffd7-638f-4940-920e-24c8ce1c15a9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:29.835266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a0cad-dd0d-4384-9bed-ad67644ae2e2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:29.846576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 792/1588] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:29.851635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a0cad-dd0d-4384-9bed-ad67644ae2e2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:29.859376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c835e-ff2e-4f72-9e67-8d1f59d8e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:29.861456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c835e-ff2e-4f72-9e67-8d1f59d8e86f admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:29.861842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c835e-ff2e-4f72-9e67-8d1f59d8e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:29.862232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c835e-ff2e-4f72-9e67-8d1f59d8e86f admin admin] PATH_INFO: `/v3/projects/3b74d5c0485641b5937875f1dc4f6959/users/6299f05d64a249cdae8edde42a669abf/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:29.884899 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c835e-ff2e-4f72-9e67-8d1f59d8e86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b74d5c0485641b5937875f1dc4f6959, user_id=6299f05d64a249cdae8edde42a669abf, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:29.888728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c835e-ff2e-4f72-9e67-8d1f59d8e86f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:29.975401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 797/1589] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:29 2022] PUT /identity/v3/projects/3b74d5c0485641b5937875f1dc4f6959/users/6299f05d64a249cdae8edde42a669abf/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:26:29.976854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383ffd7-638f-4940-920e-24c8ce1c15a9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:30.000216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383ffd7-638f-4940-920e-24c8ce1c15a9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:30.144228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b96b55-5527-4c69-ac01-c4999aa482c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:30.145095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96b55-5527-4c69-ac01-c4999aa482c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:30.145456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96b55-5527-4c69-ac01-c4999aa482c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:30.145727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96b55-5527-4c69-ac01-c4999aa482c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:30.146544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b96b55-5527-4c69-ac01-c4999aa482c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:30.148766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b96b55-5527-4c69-ac01-c4999aa482c2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:30.160308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 793/1590] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:30.164469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c835e-ff2e-4f72-9e67-8d1f59d8e86f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:30.175615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c835e-ff2e-4f72-9e67-8d1f59d8e86f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:30.183230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:30.184154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:30.184614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:30.185163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] PATH_INFO: `/v3/projects/3b74d5c0485641b5937875f1dc4f6959/users/6299f05d64a249cdae8edde42a669abf/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:30.187966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b74d5c0485641b5937875f1dc4f6959, user_id=6299f05d64a249cdae8edde42a669abf, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:30.192240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:30.208575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 798/1591] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:30 2022] PUT /identity/v3/projects/3b74d5c0485641b5937875f1dc4f6959/users/6299f05d64a249cdae8edde42a669abf/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:30.219779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e70a76-deec-4503-a73a-addd714d44ad None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:30.220157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e70a76-deec-4503-a73a-addd714d44ad None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:30.220639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e70a76-deec-4503-a73a-addd714d44ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:30.292650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e7e70a76-deec-4503-a73a-addd714d44ad None None] MFA Rules not processed for user `6299f05d64a249cdae8edde42a669abf`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:30.295080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e7e70a76-deec-4503-a73a-addd714d44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:30.524698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 794/1592] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:30.540987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:30.543723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:30.693046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327affa-7a8d-471e-98f2-c383f9a409d1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:30.700129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:30.701043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:30.703311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:30.703311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:30.717085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:30.718582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:30.729262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 799/1593] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:26:30.812548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e70a76-deec-4503-a73a-addd714d44ad None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:30.814199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e7e70a76-deec-4503-a73a-addd714d44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:30.831367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:30.863772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:30.865215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:30.877529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ac062-4997-4f1e-aca6-a9478596ab10 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:30.881253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd7e03-8ca9-4355-86c2-46a6901ae296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:30.881853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd7e03-8ca9-4355-86c2-46a6901ae296 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:30.882128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd7e03-8ca9-4355-86c2-46a6901ae296 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:30.882398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd7e03-8ca9-4355-86c2-46a6901ae296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:30.891153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e70a76-deec-4503-a73a-addd714d44ad None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:30.895753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:30.896403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:30.896687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:30.896955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] PATH_INFO: `/v3/projects/74bed532dc8e45de92f7c166baa43374` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:30.898263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74bed532dc8e45de92f7c166baa43374)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:30.901213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:30.908411 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:30.942234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd7e03-8ca9-4355-86c2-46a6901ae296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:30.943799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd7e03-8ca9-4355-86c2-46a6901ae296 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:30.953474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 800/1594] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:26:31.010543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74bed532dc8e45de92f7c166baa43374', 'request_id': 'req-b54ba01b-9e30-4732-a008-a05d17405620'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.089982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74bed532dc8e45de92f7c166baa43374', 'request_id': 'req-b54ba01b-9e30-4732-a008-a05d17405620'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.097883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74bed532dc8e45de92f7c166baa43374', 'request_id': 'req-b54ba01b-9e30-4732-a008-a05d17405620'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.116348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] The token cache is being invalidate because project 74bed532dc8e45de92f7c166baa43374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:31.116684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.117490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-b54ba01b-9e30-4732-a008-a05d17405620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.120310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 795/1595] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:30 2022] DELETE /identity/v3/projects/74bed532dc8e45de92f7c166baa43374 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:31.129794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ea86-8a16-48f6-bc6c-0c51be4b30d7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.130256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ea86-8a16-48f6-bc6c-0c51be4b30d7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.130630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65ea86-8a16-48f6-bc6c-0c51be4b30d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.178159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8d65ea86-8a16-48f6-bc6c-0c51be4b30d7 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:31.179491 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8d65ea86-8a16-48f6-bc6c-0c51be4b30d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:31.257679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 801/1596] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:31.269666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ef528-e0d6-4292-ae46-2735c51f99ef None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.269981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ef528-e0d6-4292-ae46-2735c51f99ef None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.270313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ef528-e0d6-4292-ae46-2735c51f99ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.300907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-169ef528-e0d6-4292-ae46-2735c51f99ef None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:31.302302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-169ef528-e0d6-4292-ae46-2735c51f99ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:31.314834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 796/1597] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:31.324098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65ea86-8a16-48f6-bc6c-0c51be4b30d7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:31.341507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65ea86-8a16-48f6-bc6c-0c51be4b30d7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:31.350039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:31.351728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.352209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.353309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.354387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:26:31.354903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:31.359302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:31.374343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 802/1598] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:26:31.379671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ef528-e0d6-4292-ae46-2735c51f99ef None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:31.390475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ef528-e0d6-4292-ae46-2735c51f99ef None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:31.395640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f605-56ab-41b2-851c-103adc934839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:31.396067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f605-56ab-41b2-851c-103adc934839 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.396492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f605-56ab-41b2-851c-103adc934839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.396839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f605-56ab-41b2-851c-103adc934839 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.397544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f605-56ab-41b2-851c-103adc934839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:31.400231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f605-56ab-41b2-851c-103adc934839 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:31.490111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:31.497040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 797/1599] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:31 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:26:31.497189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:31.503032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f605-56ab-41b2-851c-103adc934839 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:31.513781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc31a52-1b20-435e-bb9f-0b6b733c5f8d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:31.600381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:31.601448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.601448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.601816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] PATH_INFO: `/v3/users/81108c59dc434b459477edc7893b6fd4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.602784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81108c59dc434b459477edc7893b6fd4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:31.603643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f605-56ab-41b2-851c-103adc934839 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:31.604978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:31.609094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aba162-f6d2-4d1d-bb91-913239709cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:31.610147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba162-f6d2-4d1d-bb91-913239709cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.610429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba162-f6d2-4d1d-bb91-913239709cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.610767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aba162-f6d2-4d1d-bb91-913239709cfa admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.611545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aba162-f6d2-4d1d-bb91-913239709cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:31.613750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aba162-f6d2-4d1d-bb91-913239709cfa admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:31.704985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 798/1600] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:31 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:31.707061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aba162-f6d2-4d1d-bb91-913239709cfa admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:31.716524 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:31.720676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aba162-f6d2-4d1d-bb91-913239709cfa admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:31.726920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde81f7-0bc3-4671-920c-9cbfd5270321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:31.727775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde81f7-0bc3-4671-920c-9cbfd5270321 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.728397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde81f7-0bc3-4671-920c-9cbfd5270321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.728834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde81f7-0bc3-4671-920c-9cbfd5270321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.729619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde81f7-0bc3-4671-920c-9cbfd5270321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:31.734399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde81f7-0bc3-4671-920c-9cbfd5270321 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:31.748140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 799/1601] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:31.752388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde81f7-0bc3-4671-920c-9cbfd5270321 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:31.767035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde81f7-0bc3-4671-920c-9cbfd5270321 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:31.771908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d9f1-7061-46cd-af8f-d347f651bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:31.772705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d9f1-7061-46cd-af8f-d347f651bffc admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.773054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d9f1-7061-46cd-af8f-d347f651bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.773468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d9f1-7061-46cd-af8f-d347f651bffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.774166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186d9f1-7061-46cd-af8f-d347f651bffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:31.775764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186d9f1-7061-46cd-af8f-d347f651bffc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:31.787078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 800/1602] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:26:31.791989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d9f1-7061-46cd-af8f-d347f651bffc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:31.807897 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d9f1-7061-46cd-af8f-d347f651bffc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:31.813247 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209eb5c-d49c-49ec-b2b6-b670ec0b79c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:31.814241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209eb5c-d49c-49ec-b2b6-b670ec0b79c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.814241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209eb5c-d49c-49ec-b2b6-b670ec0b79c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.814474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209eb5c-d49c-49ec-b2b6-b670ec0b79c3 admin admin] PATH_INFO: `/v3/projects/de6d3c11508b4b388b44e9a7eb73e753/users/60ff8556cc6d4290834d3951a37f9945/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.832233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209eb5c-d49c-49ec-b2b6-b670ec0b79c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de6d3c11508b4b388b44e9a7eb73e753, user_id=60ff8556cc6d4290834d3951a37f9945, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:31.851682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209eb5c-d49c-49ec-b2b6-b670ec0b79c3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:31.865204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81108c59dc434b459477edc7893b6fd4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.873083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81108c59dc434b459477edc7893b6fd4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.875581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:31.888869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81108c59dc434b459477edc7893b6fd4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.921880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 801/1603] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:31 2022] PUT /identity/v3/projects/de6d3c11508b4b388b44e9a7eb73e753/users/60ff8556cc6d4290834d3951a37f9945/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:31.937235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c153-3a86-49cc-9827-bef764feb05e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.937733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c153-3a86-49cc-9827-bef764feb05e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.938141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644c153-3a86-49cc-9827-bef764feb05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.939444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81108c59dc434b459477edc7893b6fd4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.940484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5d93508-f794-44a0-a689-72a05cc81675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81108c59dc434b459477edc7893b6fd4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:31.968578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 803/1604] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:31 2022] DELETE /identity/v3/users/81108c59dc434b459477edc7893b6fd4 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:26:31.968714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c644c153-3a86-49cc-9827-bef764feb05e None None] MFA Rules not processed for user `60ff8556cc6d4290834d3951a37f9945`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:31.969185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c644c153-3a86-49cc-9827-bef764feb05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:31.972053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eee86-a63d-4662-ad18-a044212d7571 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:31.972460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eee86-a63d-4662-ad18-a044212d7571 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:31.972737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eee86-a63d-4662-ad18-a044212d7571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:31.994014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-de4eee86-a63d-4662-ad18-a044212d7571 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:31.995243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-de4eee86-a63d-4662-ad18-a044212d7571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:32.037627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 802/1605] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:32.042735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644c153-3a86-49cc-9827-bef764feb05e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.044455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c644c153-3a86-49cc-9827-bef764feb05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:32.056450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 804/1606] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:32.066714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eee86-a63d-4662-ad18-a044212d7571 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.068130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-de4eee86-a63d-4662-ad18-a044212d7571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:32.123482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644c153-3a86-49cc-9827-bef764feb05e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:32.125887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eee86-a63d-4662-ad18-a044212d7571 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:32.129922 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f35c1-48a4-4ae3-ad85-04b3373d631f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:32.130578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f35c1-48a4-4ae3-ad85-04b3373d631f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.130855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f35c1-48a4-4ae3-ad85-04b3373d631f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.131126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f35c1-48a4-4ae3-ad85-04b3373d631f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.132742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de019c-c0d1-48a3-aa93-07b0b31318ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:32.133833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de019c-c0d1-48a3-aa93-07b0b31318ca admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.134346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de019c-c0d1-48a3-aa93-07b0b31318ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.135125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de019c-c0d1-48a3-aa93-07b0b31318ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.136485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de019c-c0d1-48a3-aa93-07b0b31318ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:32.140478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de019c-c0d1-48a3-aa93-07b0b31318ca admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:32.141274 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f35c1-48a4-4ae3-ad85-04b3373d631f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:32.142949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f35c1-48a4-4ae3-ad85-04b3373d631f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:32.154977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 805/1607] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:26:32.242678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 803/1608] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:32 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:32.246505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f35c1-48a4-4ae3-ad85-04b3373d631f service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.259765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f35c1-48a4-4ae3-ad85-04b3373d631f service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:32.341307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e9d74-ca73-435c-9f82-2ae48cec0916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:32.341927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9d74-ca73-435c-9f82-2ae48cec0916 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.342203 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9d74-ca73-435c-9f82-2ae48cec0916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.342474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e9d74-ca73-435c-9f82-2ae48cec0916 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.343191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e9d74-ca73-435c-9f82-2ae48cec0916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:32.345302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e9d74-ca73-435c-9f82-2ae48cec0916 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:32.469107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 806/1609] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:32 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:32.470253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de019c-c0d1-48a3-aa93-07b0b31318ca admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.481547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de019c-c0d1-48a3-aa93-07b0b31318ca admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:32.486137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178a531-94b1-40a0-a3ea-b0a940a7024f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:32.486898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178a531-94b1-40a0-a3ea-b0a940a7024f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.487179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178a531-94b1-40a0-a3ea-b0a940a7024f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.487451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178a531-94b1-40a0-a3ea-b0a940a7024f admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.488246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178a531-94b1-40a0-a3ea-b0a940a7024f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:32.489914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178a531-94b1-40a0-a3ea-b0a940a7024f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:32.506454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 804/1610] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:32.507342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e9d74-ca73-435c-9f82-2ae48cec0916 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.521016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e9d74-ca73-435c-9f82-2ae48cec0916 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:32.527132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:32.527759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.528033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.528393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.528971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:32.530715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:32.532198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:32.546291 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 807/1611] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:32.551601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178a531-94b1-40a0-a3ea-b0a940a7024f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.563986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178a531-94b1-40a0-a3ea-b0a940a7024f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:32.570039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4262a-3ea3-4d7c-912b-65ae3c8b074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:32.570715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4262a-3ea3-4d7c-912b-65ae3c8b074c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.570991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4262a-3ea3-4d7c-912b-65ae3c8b074c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.571261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4262a-3ea3-4d7c-912b-65ae3c8b074c admin admin] PATH_INFO: `/v3/projects/83e459c1aa504495a7010b57cc3c5bc2/users/9f6a4ff960e641c2a849ddb926d87456/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.591647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4262a-3ea3-4d7c-912b-65ae3c8b074c admin admin] RBAC: Authorizing `identity:create_grant(project_id=83e459c1aa504495a7010b57cc3c5bc2, user_id=9f6a4ff960e641c2a849ddb926d87456, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:32.594686 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4262a-3ea3-4d7c-912b-65ae3c8b074c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:32.670738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:32.677811 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 805/1612] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:32 2022] PUT /identity/v3/projects/83e459c1aa504495a7010b57cc3c5bc2/users/9f6a4ff960e641c2a849ddb926d87456/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:32.681126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.682876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:32.698182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7916e3-9081-4474-8606-a965cc98834c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:32.701471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061b62a-203c-415a-91ac-431aee936425 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.701755 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061b62a-203c-415a-91ac-431aee936425 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.702565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061b62a-203c-415a-91ac-431aee936425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.736099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c061b62a-203c-415a-91ac-431aee936425 None None] MFA Rules not processed for user `9f6a4ff960e641c2a849ddb926d87456`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:32.737356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c061b62a-203c-415a-91ac-431aee936425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:32.764635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:32.765292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.765589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.765835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] PATH_INFO: `/v3/projects/391f4d97725b4f8280c43f4b32b3e4c1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.766919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=391f4d97725b4f8280c43f4b32b3e4c1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:32.768839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:32.803397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 806/1613] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:32.807802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061b62a-203c-415a-91ac-431aee936425 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.818289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061b62a-203c-415a-91ac-431aee936425 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:32.823312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46aa06-8aca-4626-915c-dcce87e0c88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:32.823893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46aa06-8aca-4626-915c-dcce87e0c88c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:32.824170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46aa06-8aca-4626-915c-dcce87e0c88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:32.824471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46aa06-8aca-4626-915c-dcce87e0c88c admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:32.825089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46aa06-8aca-4626-915c-dcce87e0c88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:32.826969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46aa06-8aca-4626-915c-dcce87e0c88c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:32.859043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391f4d97725b4f8280c43f4b32b3e4c1', 'request_id': 'req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:32.864706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391f4d97725b4f8280c43f4b32b3e4c1', 'request_id': 'req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:32.871653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391f4d97725b4f8280c43f4b32b3e4c1', 'request_id': 'req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:32.909825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 807/1614] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:32 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:26:32.910878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46aa06-8aca-4626-915c-dcce87e0c88c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.934316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] The token cache is being invalidate because project 391f4d97725b4f8280c43f4b32b3e4c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:32.934616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:32.936995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:32.938562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 808/1615] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:32 2022] DELETE /identity/v3/projects/391f4d97725b4f8280c43f4b32b3e4c1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:32.943981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:32.945948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:32.997914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46aa06-8aca-4626-915c-dcce87e0c88c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:33.005015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:33.006307 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.008698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.009139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.009995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:33.016719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:33.073203 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cf26a-4f15-4ad3-9e1a-2f3e4ce94f5d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:33.093986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:33.094644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.094915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.095189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] PATH_INFO: `/v3/users/6299f05d64a249cdae8edde42a669abf` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.096401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6299f05d64a249cdae8edde42a669abf)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:33.098668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:33.145814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 808/1616] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:32 2022] POST /identity/v3/users => generated 295 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:33.158907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:33.161062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:33.175823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac89b9-3956-4233-a41c-2e9e46e049a0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:33.182196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12f64e-687d-47d4-b3b0-c8b2dd3f2832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:33.182793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12f64e-687d-47d4-b3b0-c8b2dd3f2832 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.183070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12f64e-687d-47d4-b3b0-c8b2dd3f2832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.183684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12f64e-687d-47d4-b3b0-c8b2dd3f2832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.184467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12f64e-687d-47d4-b3b0-c8b2dd3f2832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:33.186050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12f64e-687d-47d4-b3b0-c8b2dd3f2832 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:33.196655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 809/1617] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:26:33.201034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12f64e-687d-47d4-b3b0-c8b2dd3f2832 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:33.203524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:33.210765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12f64e-687d-47d4-b3b0-c8b2dd3f2832 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:33.215272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fc17a-bc90-4652-bba7-0c73fa273a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:33.215895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fc17a-bc90-4652-bba7-0c73fa273a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.216176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fc17a-bc90-4652-bba7-0c73fa273a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.216483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fc17a-bc90-4652-bba7-0c73fa273a2a admin admin] PATH_INFO: `/v3/projects/334d1fceb8514d90959036e927f7bf67/users/51dc16f482944951a3017f4d76aea6f8/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.231472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fc17a-bc90-4652-bba7-0c73fa273a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=334d1fceb8514d90959036e927f7bf67, user_id=51dc16f482944951a3017f4d76aea6f8, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:33.234253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fc17a-bc90-4652-bba7-0c73fa273a2a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:33.314574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6299f05d64a249cdae8edde42a669abf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:33.322454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6299f05d64a249cdae8edde42a669abf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:33.333503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 810/1618] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:33 2022] PUT /identity/v3/projects/334d1fceb8514d90959036e927f7bf67/users/51dc16f482944951a3017f4d76aea6f8/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:26:33.337401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fc17a-bc90-4652-bba7-0c73fa273a2a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:33.339655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6299f05d64a249cdae8edde42a669abf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:33.359538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:33.401026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fc17a-bc90-4652-bba7-0c73fa273a2a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:33.406095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3941c-8807-4609-8078-cf600ab5133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:33.406709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3941c-8807-4609-8078-cf600ab5133e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.406990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3941c-8807-4609-8078-cf600ab5133e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.407358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3941c-8807-4609-8078-cf600ab5133e admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.407895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3941c-8807-4609-8078-cf600ab5133e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:33.409505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3941c-8807-4609-8078-cf600ab5133e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:33.422999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 811/1619] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:26:33.423917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3941c-8807-4609-8078-cf600ab5133e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:33.434284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3941c-8807-4609-8078-cf600ab5133e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:33.439463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:33.441623 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6299f05d64a249cdae8edde42a669abf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:33.442245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c76e32d5-d405-481a-945d-9f9f75b9ef2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6299f05d64a249cdae8edde42a669abf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:33.445277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.450308 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 809/1620] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:32 2022] DELETE /identity/v3/users/6299f05d64a249cdae8edde42a669abf => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:33.450473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.450473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] PATH_INFO: `/v3/projects/334d1fceb8514d90959036e927f7bf67/users/51dc16f482944951a3017f4d76aea6f8/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.453834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] RBAC: Authorizing `identity:create_grant(project_id=334d1fceb8514d90959036e927f7bf67, user_id=51dc16f482944951a3017f4d76aea6f8, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:33.456369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:33.469736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 812/1621] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:33 2022] PUT /identity/v3/projects/334d1fceb8514d90959036e927f7bf67/users/51dc16f482944951a3017f4d76aea6f8/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:33.486908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac714-a77c-4055-8b4d-e64fa92e77ee None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.488350 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac714-a77c-4055-8b4d-e64fa92e77ee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.488653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fac714-a77c-4055-8b4d-e64fa92e77ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.523392 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-71fac714-a77c-4055-8b4d-e64fa92e77ee None None] MFA Rules not processed for user `51dc16f482944951a3017f4d76aea6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:33.524880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-71fac714-a77c-4055-8b4d-e64fa92e77ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:33.606171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 810/1622] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:33.618985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:33.620687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:33.685958 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9a866-4253-46f0-89f6-0edfda2002be admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:33.689759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5144c5c-677c-4b6a-8900-f68d39218c53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:33.690362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5144c5c-677c-4b6a-8900-f68d39218c53 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.690637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5144c5c-677c-4b6a-8900-f68d39218c53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.690905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5144c5c-677c-4b6a-8900-f68d39218c53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.692803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c5144c5c-677c-4b6a-8900-f68d39218c53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:33.770303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5144c5c-677c-4b6a-8900-f68d39218c53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:33.771989 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5144c5c-677c-4b6a-8900-f68d39218c53 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:33.781638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 813/1623] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:33.889345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fac714-a77c-4055-8b4d-e64fa92e77ee None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:33.891576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-71fac714-a77c-4055-8b4d-e64fa92e77ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:33.980929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fac714-a77c-4055-8b4d-e64fa92e77ee None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:33.985412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:33.986029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:33.986306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:33.986573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] PATH_INFO: `/v3/projects/3b74d5c0485641b5937875f1dc4f6959` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:33.987663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b74d5c0485641b5937875f1dc4f6959)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:33.989685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:34.111439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:34.134718 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b74d5c0485641b5937875f1dc4f6959', 'request_id': 'req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:34.141094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b74d5c0485641b5937875f1dc4f6959', 'request_id': 'req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:34.148329 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b74d5c0485641b5937875f1dc4f6959', 'request_id': 'req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:34.245394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] The token cache is being invalidate because project 3b74d5c0485641b5937875f1dc4f6959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:34.245981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:34.247072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34a3bdfc-7bb2-4a1f-b379-86de4a9787dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:34.253174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 811/1624] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:33 2022] DELETE /identity/v3/projects/3b74d5c0485641b5937875f1dc4f6959 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:34.254615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0e043-682f-4b79-9940-57c5c2ca1d5a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:34.254913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0e043-682f-4b79-9940-57c5c2ca1d5a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:34.255456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0e043-682f-4b79-9940-57c5c2ca1d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:34.280878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-27e0e043-682f-4b79-9940-57c5c2ca1d5a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:34.282067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-27e0e043-682f-4b79-9940-57c5c2ca1d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:34.338749 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:34.348584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 814/1625] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:34.356181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d114a2-2ead-4da8-b3d0-55cf9d8f18db None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:34.356495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d114a2-2ead-4da8-b3d0-55cf9d8f18db None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:34.356774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d114a2-2ead-4da8-b3d0-55cf9d8f18db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:34.377220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-36d114a2-2ead-4da8-b3d0-55cf9d8f18db None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:34.378416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-36d114a2-2ead-4da8-b3d0-55cf9d8f18db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:34.387095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 812/1626] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:34.391308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0e043-682f-4b79-9940-57c5c2ca1d5a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:34.401871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0e043-682f-4b79-9940-57c5c2ca1d5a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:34.413481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:34.413481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:34.413481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:34.413481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:34.413778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:26:34.413778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:34.413778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:34.436048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 815/1627] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:26:34.446592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d114a2-2ead-4da8-b3d0-55cf9d8f18db None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:34.457327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d114a2-2ead-4da8-b3d0-55cf9d8f18db None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:34.463274 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe7034-87e4-478f-b0e3-05740c2fa615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:34.463964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe7034-87e4-478f-b0e3-05740c2fa615 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:34.464243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe7034-87e4-478f-b0e3-05740c2fa615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:34.464564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe7034-87e4-478f-b0e3-05740c2fa615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:34.470541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe7034-87e4-478f-b0e3-05740c2fa615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:34.470690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe7034-87e4-478f-b0e3-05740c2fa615 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:34.611490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 813/1628] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:34 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:26:34.613324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:34.623301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654ada6-4b0a-44bd-b008-165da20f6ecd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:34.690816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52163ca8-8026-4cb9-a82b-11cc687008ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:34.702184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52163ca8-8026-4cb9-a82b-11cc687008ef admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:34.702184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52163ca8-8026-4cb9-a82b-11cc687008ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:34.702184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52163ca8-8026-4cb9-a82b-11cc687008ef admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:34.702184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52163ca8-8026-4cb9-a82b-11cc687008ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:34.702184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52163ca8-8026-4cb9-a82b-11cc687008ef admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:34.801441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 816/1629] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:34 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:34.806069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe7034-87e4-478f-b0e3-05740c2fa615 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:34.816568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe7034-87e4-478f-b0e3-05740c2fa615 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:34.821891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656d0b-e6d5-4bcb-8066-c33855b773b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:34.822521 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656d0b-e6d5-4bcb-8066-c33855b773b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:34.822802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656d0b-e6d5-4bcb-8066-c33855b773b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:34.823074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656d0b-e6d5-4bcb-8066-c33855b773b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:34.823716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656d0b-e6d5-4bcb-8066-c33855b773b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:34.825336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656d0b-e6d5-4bcb-8066-c33855b773b4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:34.835576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 814/1630] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:26:34.839927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52163ca8-8026-4cb9-a82b-11cc687008ef admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:34.849840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52163ca8-8026-4cb9-a82b-11cc687008ef admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:34.854356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9449b9-91eb-4b4f-af4c-366434de2df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:34.854941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9449b9-91eb-4b4f-af4c-366434de2df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:34.855220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9449b9-91eb-4b4f-af4c-366434de2df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:34.855493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9449b9-91eb-4b4f-af4c-366434de2df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:34.856121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9449b9-91eb-4b4f-af4c-366434de2df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:34.857743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9449b9-91eb-4b4f-af4c-366434de2df3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:34.868803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 817/1631] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:34.872687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656d0b-e6d5-4bcb-8066-c33855b773b4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:34.883638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656d0b-e6d5-4bcb-8066-c33855b773b4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:34.888542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe948-8cff-4d9f-9f39-5bb080d0ea1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:34.891018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe948-8cff-4d9f-9f39-5bb080d0ea1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:34.891018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe948-8cff-4d9f-9f39-5bb080d0ea1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:34.891018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe948-8cff-4d9f-9f39-5bb080d0ea1e admin admin] PATH_INFO: `/v3/projects/dedd63f8230c4c92af0f14567f0c77eb/users/9f476f9eb76546f18408c6784ecb828c/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:34.904556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fe948-8cff-4d9f-9f39-5bb080d0ea1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dedd63f8230c4c92af0f14567f0c77eb, user_id=9f476f9eb76546f18408c6784ecb828c, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:34.907161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fe948-8cff-4d9f-9f39-5bb080d0ea1e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:34.996223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 815/1632] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:34 2022] PUT /identity/v3/projects/dedd63f8230c4c92af0f14567f0c77eb/users/9f476f9eb76546f18408c6784ecb828c/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:26:35.008337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc0af6-1f91-469f-b21c-d9ee0b9a7f30 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:35.008629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc0af6-1f91-469f-b21c-d9ee0b9a7f30 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:35.008907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc0af6-1f91-469f-b21c-d9ee0b9a7f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:35.052365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3acc0af6-1f91-469f-b21c-d9ee0b9a7f30 None None] MFA Rules not processed for user `9f476f9eb76546f18408c6784ecb828c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:35.053592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3acc0af6-1f91-469f-b21c-d9ee0b9a7f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:35.134983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 818/1633] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:35.139347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe948-8cff-4d9f-9f39-5bb080d0ea1e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:35.206630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe948-8cff-4d9f-9f39-5bb080d0ea1e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:35.211729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa736a-2ba2-4589-b74e-4bdc2bd62bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:35.212532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa736a-2ba2-4589-b74e-4bdc2bd62bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:35.212882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa736a-2ba2-4589-b74e-4bdc2bd62bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:35.213221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa736a-2ba2-4589-b74e-4bdc2bd62bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:35.213953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa736a-2ba2-4589-b74e-4bdc2bd62bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:35.216066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa736a-2ba2-4589-b74e-4bdc2bd62bb5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:35.318311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 816/1634] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:26:35.322200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc0af6-1f91-469f-b21c-d9ee0b9a7f30 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:35.334041 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc0af6-1f91-469f-b21c-d9ee0b9a7f30 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:35.389253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:35.398712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c78241-2fc3-41aa-a4a7-5740c70e2534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:35.399987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c78241-2fc3-41aa-a4a7-5740c70e2534 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:35.399987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c78241-2fc3-41aa-a4a7-5740c70e2534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:35.399987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c78241-2fc3-41aa-a4a7-5740c70e2534 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:35.400609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c78241-2fc3-41aa-a4a7-5740c70e2534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:35.402986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c78241-2fc3-41aa-a4a7-5740c70e2534 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:35.548304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 819/1635] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:35 2022] POST /identity/v3/users => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:26:35.553576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa736a-2ba2-4589-b74e-4bdc2bd62bb5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:35.565498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa736a-2ba2-4589-b74e-4bdc2bd62bb5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:35.570304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739d4ad-ff2e-42f6-902b-d25de164253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:35.570980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d4ad-ff2e-42f6-902b-d25de164253b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:35.572431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d4ad-ff2e-42f6-902b-d25de164253b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:35.572431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739d4ad-ff2e-42f6-902b-d25de164253b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:35.572431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739d4ad-ff2e-42f6-902b-d25de164253b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:35.573802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739d4ad-ff2e-42f6-902b-d25de164253b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:35.584816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 817/1636] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:26:35.589820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c78241-2fc3-41aa-a4a7-5740c70e2534 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:35.600170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c78241-2fc3-41aa-a4a7-5740c70e2534 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:35.605537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bad3c8-6e28-43b7-a08c-21eaaad706f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:35.606209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bad3c8-6e28-43b7-a08c-21eaaad706f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:35.606489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bad3c8-6e28-43b7-a08c-21eaaad706f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:35.606761 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bad3c8-6e28-43b7-a08c-21eaaad706f4 admin admin] PATH_INFO: `/v3/projects/349bbad6d48b469db5da2551f815bb9a/users/1055e57ac1a54169babc554452e22258/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:35.622715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bad3c8-6e28-43b7-a08c-21eaaad706f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349bbad6d48b469db5da2551f815bb9a, user_id=1055e57ac1a54169babc554452e22258, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:35.625615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bad3c8-6e28-43b7-a08c-21eaaad706f4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:35.693178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 820/1637] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:35 2022] PUT /identity/v3/projects/349bbad6d48b469db5da2551f815bb9a/users/1055e57ac1a54169babc554452e22258/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:35.697260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739d4ad-ff2e-42f6-902b-d25de164253b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:35.707471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739d4ad-ff2e-42f6-902b-d25de164253b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:35.738107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:35.766248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ec500-3895-46f3-89ea-dc85b5a6fb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:35.766915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ec500-3895-46f3-89ea-dc85b5a6fb38 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:35.767249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ec500-3895-46f3-89ea-dc85b5a6fb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:35.767523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ec500-3895-46f3-89ea-dc85b5a6fb38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:35.768161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ec500-3895-46f3-89ea-dc85b5a6fb38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:35.769811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ec500-3895-46f3-89ea-dc85b5a6fb38 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:35.780957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 818/1638] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:35.785340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bad3c8-6e28-43b7-a08c-21eaaad706f4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:35.796446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bad3c8-6e28-43b7-a08c-21eaaad706f4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:35.801290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:35.801949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:35.802873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:35.802873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] PATH_INFO: `/v3/projects/349bbad6d48b469db5da2551f815bb9a/users/1055e57ac1a54169babc554452e22258/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:35.807469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] RBAC: Authorizing `identity:create_grant(project_id=349bbad6d48b469db5da2551f815bb9a, user_id=1055e57ac1a54169babc554452e22258, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:35.810338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:35.822998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 821/1639] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:35 2022] PUT /identity/v3/projects/349bbad6d48b469db5da2551f815bb9a/users/1055e57ac1a54169babc554452e22258/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:35.834187 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979debf-a6d4-4730-a966-905993f43647 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:35.834494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979debf-a6d4-4730-a966-905993f43647 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:35.834772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979debf-a6d4-4730-a966-905993f43647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:35.865206 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a979debf-a6d4-4730-a966-905993f43647 None None] MFA Rules not processed for user `1055e57ac1a54169babc554452e22258`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:35.866698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a979debf-a6d4-4730-a966-905993f43647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:35.958391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 819/1640] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:35.970104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:35.979219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:36.040718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94007fe-8647-4132-8eaf-99c9377b811c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:36.044375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cd268-c08b-448a-b738-1b4fa0a28271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:36.044978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cd268-c08b-448a-b738-1b4fa0a28271 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:36.045346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cd268-c08b-448a-b738-1b4fa0a28271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:36.045622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cd268-c08b-448a-b738-1b4fa0a28271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:36.121516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cd268-c08b-448a-b738-1b4fa0a28271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:36.122995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cd268-c08b-448a-b738-1b4fa0a28271 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:36.133654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 822/1641] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:36.864783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979debf-a6d4-4730-a966-905993f43647 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:36.867375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a979debf-a6d4-4730-a966-905993f43647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:36.879292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979debf-a6d4-4730-a966-905993f43647 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:36.882811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:36.883408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:36.883692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:36.883962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:36.885673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:36.950096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:36.951609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:36.962000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 820/1642] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:38.676432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cd268-c08b-448a-b738-1b4fa0a28271 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:38.685614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cd268-c08b-448a-b738-1b4fa0a28271 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:38.690062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:38.690671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:38.690945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:38.691211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:38.692976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:38.770450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:38.775388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:38.777044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:38.788958 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 823/1643] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:42.414091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:42.422421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef034b8-27e2-4483-8348-21a652c9955d service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:42.426023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9126d95-d945-471e-8ea8-4dd6025539a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:42.426610 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9126d95-d945-471e-8ea8-4dd6025539a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:42.426881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9126d95-d945-471e-8ea8-4dd6025539a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:42.427143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9126d95-d945-471e-8ea8-4dd6025539a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:42.435966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9126d95-d945-471e-8ea8-4dd6025539a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:42.437397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9126d95-d945-471e-8ea8-4dd6025539a3 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:42.446503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 821/1644] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:49.012304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:52.442429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=83 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:53.333885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:53.335978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:53.411462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c0c67-ab84-4d14-9d8f-c5049fd9e11c service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:53.416601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:53.417233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:53.417518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:53.417787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] PATH_INFO: `/v3/users/60ff8556cc6d4290834d3951a37f9945` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:53.418910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ff8556cc6d4290834d3951a37f9945)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:53.421440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:53.552966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ff8556cc6d4290834d3951a37f9945'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:53.554437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ff8556cc6d4290834d3951a37f9945'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:53.568507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ff8556cc6d4290834d3951a37f9945'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:53.580768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ff8556cc6d4290834d3951a37f9945'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:53.582816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ff8556cc6d4290834d3951a37f9945'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:53.686647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 824/1645] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:53 2022] DELETE /identity/v3/users/60ff8556cc6d4290834d3951a37f9945 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:53.691151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24696ea3-d71a-497d-87e4-e8d7f1830b40 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:53.691656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24696ea3-d71a-497d-87e4-e8d7f1830b40 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:53.691752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24696ea3-d71a-497d-87e4-e8d7f1830b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:53.712263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-24696ea3-d71a-497d-87e4-e8d7f1830b40 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:53.713535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-24696ea3-d71a-497d-87e4-e8d7f1830b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:53.774435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 822/1646] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:53.784324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:53.785731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:53.844172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ecdaa-a211-4e45-b07c-dcd2cf3c055d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:53.847971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3178558-f6d1-4d01-a6e7-c108bc4cdeb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:53.848607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3178558-f6d1-4d01-a6e7-c108bc4cdeb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:53.849008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3178558-f6d1-4d01-a6e7-c108bc4cdeb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:53.849284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3178558-f6d1-4d01-a6e7-c108bc4cdeb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:53.860196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3178558-f6d1-4d01-a6e7-c108bc4cdeb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:53.861689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3178558-f6d1-4d01-a6e7-c108bc4cdeb9 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:53.872301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 825/1647] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:54.339895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24696ea3-d71a-497d-87e4-e8d7f1830b40 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:54.346890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-24696ea3-d71a-497d-87e4-e8d7f1830b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:54.355429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24696ea3-d71a-497d-87e4-e8d7f1830b40 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:54.360650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:54.361287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:54.361718 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:54.362083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] PATH_INFO: `/v3/projects/de6d3c11508b4b388b44e9a7eb73e753` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:54.363130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de6d3c11508b4b388b44e9a7eb73e753)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:54.365068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:54.514676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de6d3c11508b4b388b44e9a7eb73e753', 'request_id': 'req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.520326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de6d3c11508b4b388b44e9a7eb73e753', 'request_id': 'req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.527744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de6d3c11508b4b388b44e9a7eb73e753', 'request_id': 'req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.657419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] The token cache is being invalidate because project de6d3c11508b4b388b44e9a7eb73e753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:54.657708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.658418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.660148 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 823/1648] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:54 2022] DELETE /identity/v3/projects/de6d3c11508b4b388b44e9a7eb73e753 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:54.664463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3178558-f6d1-4d01-a6e7-c108bc4cdeb9 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:54.675576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3178558-f6d1-4d01-a6e7-c108bc4cdeb9 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:54.696122 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:54.736026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:54.736714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:54.736987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:54.737253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] PATH_INFO: `/v3/users/9f6a4ff960e641c2a849ddb926d87456` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:54.738407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f6a4ff960e641c2a849ddb926d87456)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:54.740649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:54.906905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f6a4ff960e641c2a849ddb926d87456'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.907614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f6a4ff960e641c2a849ddb926d87456'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.924047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f6a4ff960e641c2a849ddb926d87456'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.934098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f6a4ff960e641c2a849ddb926d87456'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.934817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f6a4ff960e641c2a849ddb926d87456'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:54.984613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 826/1649] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:54 2022] DELETE /identity/v3/users/9f6a4ff960e641c2a849ddb926d87456 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:55.433139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:55.436690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:55.487613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5930eb-0508-42a5-bbbc-69ad01bc86d4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:55.597293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:55.597929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:55.598204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:55.599028 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] PATH_INFO: `/v3/projects/83e459c1aa504495a7010b57cc3c5bc2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:55.599707 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=83e459c1aa504495a7010b57cc3c5bc2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:55.602519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:55.702814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:55.711996 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83e459c1aa504495a7010b57cc3c5bc2', 'request_id': 'req-f831aae0-0cd9-4aab-988e-dac8f22e60ca'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:55.718461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83e459c1aa504495a7010b57cc3c5bc2', 'request_id': 'req-f831aae0-0cd9-4aab-988e-dac8f22e60ca'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:55.728373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83e459c1aa504495a7010b57cc3c5bc2', 'request_id': 'req-f831aae0-0cd9-4aab-988e-dac8f22e60ca'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:55.813425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] The token cache is being invalidate because project 83e459c1aa504495a7010b57cc3c5bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:55.813787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:55.814557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:55.816429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 824/1650] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:55 2022] DELETE /identity/v3/projects/83e459c1aa504495a7010b57cc3c5bc2 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:55.820447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:55.822431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:55.838637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34905097-11e9-4574-ab20-d7792903f881 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:55.911257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:55.911908 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:55.912186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:55.912508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] PATH_INFO: `/v3/users/51dc16f482944951a3017f4d76aea6f8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:55.914277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51dc16f482944951a3017f4d76aea6f8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:55.916230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:56.120652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:56.127906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51dc16f482944951a3017f4d76aea6f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:56.128693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51dc16f482944951a3017f4d76aea6f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:56.143918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51dc16f482944951a3017f4d76aea6f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:56.151683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51dc16f482944951a3017f4d76aea6f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:56.153593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-707257a5-71f2-4222-aa1a-48b12cd9cfb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51dc16f482944951a3017f4d76aea6f8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:56.252306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 827/1651] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:55 2022] DELETE /identity/v3/users/51dc16f482944951a3017f4d76aea6f8 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:26:56.744864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:56.746418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:56.760402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831aae0-0cd9-4aab-988e-dac8f22e60ca admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:56.843850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:56.844824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:56.845204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:56.845578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] PATH_INFO: `/v3/projects/334d1fceb8514d90959036e927f7bf67` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:56.847045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=334d1fceb8514d90959036e927f7bf67)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:56.849223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:56.951876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '334d1fceb8514d90959036e927f7bf67', 'request_id': 'req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:56.958229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '334d1fceb8514d90959036e927f7bf67', 'request_id': 'req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:56.966487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '334d1fceb8514d90959036e927f7bf67', 'request_id': 'req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:57.052647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] The token cache is being invalidate because project 334d1fceb8514d90959036e927f7bf67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:57.053019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:57.058357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:57.058357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 825/1652] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:56 2022] DELETE /identity/v3/projects/334d1fceb8514d90959036e927f7bf67 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:57.060606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177af4-b99c-45f8-bb7b-52bbbd809203 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.061105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177af4-b99c-45f8-bb7b-52bbbd809203 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.061213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177af4-b99c-45f8-bb7b-52bbbd809203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.068334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:57.070527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:57.083678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a8177af4-b99c-45f8-bb7b-52bbbd809203 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:57.085655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a8177af4-b99c-45f8-bb7b-52bbbd809203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:57.139633 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:57.160356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 828/1653] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:57.160556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f1bf-f31b-49b8-868f-e4283bf72d7a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:57.165461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:57.166230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.166790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.167068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] PATH_INFO: `/v3/users/9f476f9eb76546f18408c6784ecb828c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.167815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929e1b-4a2c-4b6e-9070-2516858af659 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.168106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929e1b-4a2c-4b6e-9070-2516858af659 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.168217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f476f9eb76546f18408c6784ecb828c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:57.168415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929e1b-4a2c-4b6e-9070-2516858af659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.170138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:57.191625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-95929e1b-4a2c-4b6e-9070-2516858af659 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:57.192874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-95929e1b-4a2c-4b6e-9070-2516858af659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:57.201243 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 829/1654] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:57.205674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95929e1b-4a2c-4b6e-9070-2516858af659 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:57.215677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95929e1b-4a2c-4b6e-9070-2516858af659 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:57.220743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545757-b6f8-4333-b848-5086480649db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:57.221550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545757-b6f8-4333-b848-5086480649db admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.224983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545757-b6f8-4333-b848-5086480649db admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.224983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545757-b6f8-4333-b848-5086480649db admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.224983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545757-b6f8-4333-b848-5086480649db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:26:57.224983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545757-b6f8-4333-b848-5086480649db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:57.224983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545757-b6f8-4333-b848-5086480649db admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:57.237165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 830/1655] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:26:57.242427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545757-b6f8-4333-b848-5086480649db admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:57.252620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545757-b6f8-4333-b848-5086480649db admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:57.257939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecc53d-e58b-46b1-ab0b-e99fd61514a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:57.258525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecc53d-e58b-46b1-ab0b-e99fd61514a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.258800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecc53d-e58b-46b1-ab0b-e99fd61514a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.259278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecc53d-e58b-46b1-ab0b-e99fd61514a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.259869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecc53d-e58b-46b1-ab0b-e99fd61514a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:57.262460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecc53d-e58b-46b1-ab0b-e99fd61514a5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:57.351856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 831/1656] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:26:57 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:26:57.355829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecc53d-e58b-46b1-ab0b-e99fd61514a5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:57.368358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f476f9eb76546f18408c6784ecb828c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:57.376455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f476f9eb76546f18408c6784ecb828c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:57.408900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f476f9eb76546f18408c6784ecb828c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:57.434770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecc53d-e58b-46b1-ab0b-e99fd61514a5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:57.440222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:57.442774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.442774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.442774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.442774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:57.445035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:57.458111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f476f9eb76546f18408c6784ecb828c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:57.458757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d11feca5-a030-4a82-8ab2-6f236fdf25eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f476f9eb76546f18408c6784ecb828c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:57.466346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 826/1657] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:57 2022] DELETE /identity/v3/users/9f476f9eb76546f18408c6784ecb828c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:26:57.470816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f413258-5b42-4e18-a472-11f79239b774 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.471103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f413258-5b42-4e18-a472-11f79239b774 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.471382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f413258-5b42-4e18-a472-11f79239b774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.492973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7f413258-5b42-4e18-a472-11f79239b774 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:57.494104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7f413258-5b42-4e18-a472-11f79239b774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:57.533638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 832/1658] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:26:57 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:26:57.538631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:57.541935 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:57.557425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aff3ff-338a-4af6-af61-5750d7772367 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:57.562642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:57.568198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 827/1659] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:26:57.576379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f413258-5b42-4e18-a472-11f79239b774 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:57.579313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7f413258-5b42-4e18-a472-11f79239b774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:57.644121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7d619-8749-48ec-ad69-628fdc3d9618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:57.644760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d619-8749-48ec-ad69-628fdc3d9618 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.645036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d619-8749-48ec-ad69-628fdc3d9618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.645441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d619-8749-48ec-ad69-628fdc3d9618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.646084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7d619-8749-48ec-ad69-628fdc3d9618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:57.648394 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7d619-8749-48ec-ad69-628fdc3d9618 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:57.668379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 833/1660] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:26:57.674827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7d619-8749-48ec-ad69-628fdc3d9618 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:57.691899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7d619-8749-48ec-ad69-628fdc3d9618 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:57.700342 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93579d97-08ed-4ed6-853f-0edf0fdff6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:57.701168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579d97-08ed-4ed6-853f-0edf0fdff6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.701437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579d97-08ed-4ed6-853f-0edf0fdff6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.701704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93579d97-08ed-4ed6-853f-0edf0fdff6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.702058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f413258-5b42-4e18-a472-11f79239b774 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:57.702389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93579d97-08ed-4ed6-853f-0edf0fdff6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:57.711672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93579d97-08ed-4ed6-853f-0edf0fdff6f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:57.712221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b670eb-f0d8-451d-9641-d5c0d0d14849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:57.712896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b670eb-f0d8-451d-9641-d5c0d0d14849 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.713173 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b670eb-f0d8-451d-9641-d5c0d0d14849 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.713448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b670eb-f0d8-451d-9641-d5c0d0d14849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.733686 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 834/1661] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:26:57.734887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93579d97-08ed-4ed6-853f-0edf0fdff6f7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:57.736784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b670eb-f0d8-451d-9641-d5c0d0d14849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:57.738564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b670eb-f0d8-451d-9641-d5c0d0d14849 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:57.749223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93579d97-08ed-4ed6-853f-0edf0fdff6f7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:57.755336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:57.755643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.755915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.756890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] PATH_INFO: `/v3/projects/2eb16832d043491bbdf675ca6fb9f4cd/users/34fe65b1505d4c11991ebcf3a0cd4259/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.771094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 828/1662] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:57.775540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb16832d043491bbdf675ca6fb9f4cd, user_id=34fe65b1505d4c11991ebcf3a0cd4259, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:57.778257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:57.877170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 835/1663] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:26:57 2022] PUT /identity/v3/projects/2eb16832d043491bbdf675ca6fb9f4cd/users/34fe65b1505d4c11991ebcf3a0cd4259/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:57.889001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba519a1-021e-4a96-a838-76d7c1a03db6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:57.889557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba519a1-021e-4a96-a838-76d7c1a03db6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:57.889800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba519a1-021e-4a96-a838-76d7c1a03db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:57.904240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:57.919836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8ba519a1-021e-4a96-a838-76d7c1a03db6 None None] MFA Rules not processed for user `34fe65b1505d4c11991ebcf3a0cd4259`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:57.921047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8ba519a1-021e-4a96-a838-76d7c1a03db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:58.062618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 829/1664] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:58.076464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:58.079200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:58.167585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afeaaf-207f-445b-8e55-6bf845a3c686 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:58.171380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516194eb-a365-4781-bb59-fe13b162a843 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:58.172551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516194eb-a365-4781-bb59-fe13b162a843 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:58.172831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516194eb-a365-4781-bb59-fe13b162a843 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:58.176114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516194eb-a365-4781-bb59-fe13b162a843 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:58.184412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516194eb-a365-4781-bb59-fe13b162a843 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:58.185894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516194eb-a365-4781-bb59-fe13b162a843 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:58.198165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 836/1665] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:26:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:26:58.333886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba519a1-021e-4a96-a838-76d7c1a03db6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:58.340668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8ba519a1-021e-4a96-a838-76d7c1a03db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:58.501814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba519a1-021e-4a96-a838-76d7c1a03db6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:58.508139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:58.508907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:58.509106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:58.509376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] PATH_INFO: `/v3/projects/dedd63f8230c4c92af0f14567f0c77eb` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:58.510515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dedd63f8230c4c92af0f14567f0c77eb)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:58.513213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:58.631078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dedd63f8230c4c92af0f14567f0c77eb', 'request_id': 'req-01c4504f-8149-4047-b9ca-afd6a054864a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:58.637067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dedd63f8230c4c92af0f14567f0c77eb', 'request_id': 'req-01c4504f-8149-4047-b9ca-afd6a054864a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:58.645452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dedd63f8230c4c92af0f14567f0c77eb', 'request_id': 'req-01c4504f-8149-4047-b9ca-afd6a054864a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:58.732647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] The token cache is being invalidate because project dedd63f8230c4c92af0f14567f0c77eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:58.733304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:58.733904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:58.735481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 830/1666] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:58 2022] DELETE /identity/v3/projects/dedd63f8230c4c92af0f14567f0c77eb => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:58.739794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516194eb-a365-4781-bb59-fe13b162a843 service glance] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:58.751310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516194eb-a365-4781-bb59-fe13b162a843 service glance] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:58.826185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:58.826185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:58.826485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:58.826756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] PATH_INFO: `/v3/users/1055e57ac1a54169babc554452e22258` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:58.829098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1055e57ac1a54169babc554452e22258)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:58.831298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:58.856510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:59.053677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1055e57ac1a54169babc554452e22258'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.056717 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1055e57ac1a54169babc554452e22258'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.070971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1055e57ac1a54169babc554452e22258'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.077964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1055e57ac1a54169babc554452e22258'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.078723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bc482eb3-c6ba-4d3d-b2c8-d0d491a098dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1055e57ac1a54169babc554452e22258'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.163528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 837/1667] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:26:58 2022] DELETE /identity/v3/users/1055e57ac1a54169babc554452e22258 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:59.557071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:26:59.558544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:59.585982 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4504f-8149-4047-b9ca-afd6a054864a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:26:59.613140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:26:59.651380 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:26:59.656796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:59.656796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:59.656796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] PATH_INFO: `/v3/projects/349bbad6d48b469db5da2551f815bb9a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:59.656796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=349bbad6d48b469db5da2551f815bb9a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:26:59.656796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:26:59.773720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349bbad6d48b469db5da2551f815bb9a', 'request_id': 'req-243b824e-901f-4033-9fdb-fbde83e470d1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.773720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349bbad6d48b469db5da2551f815bb9a', 'request_id': 'req-243b824e-901f-4033-9fdb-fbde83e470d1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.779671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349bbad6d48b469db5da2551f815bb9a', 'request_id': 'req-243b824e-901f-4033-9fdb-fbde83e470d1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.851467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] The token cache is being invalidate because project 349bbad6d48b469db5da2551f815bb9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:26:59.851764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.852470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-243b824e-901f-4033-9fdb-fbde83e470d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:26:59.853999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 831/1668] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:26:59 2022] DELETE /identity/v3/projects/349bbad6d48b469db5da2551f815bb9a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:26:59.861753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b13f-3764-4721-8461-2bafd95bd45d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:59.862128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b13f-3764-4721-8461-2bafd95bd45d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:59.862480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b13f-3764-4721-8461-2bafd95bd45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:59.883291 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ca17b13f-3764-4721-8461-2bafd95bd45d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:59.884553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ca17b13f-3764-4721-8461-2bafd95bd45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:26:59.967170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 838/1669] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:26:59.977039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28402334-f35f-4fc9-8f6e-0820ee099da7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:26:59.977323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28402334-f35f-4fc9-8f6e-0820ee099da7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:26:59.977602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28402334-f35f-4fc9-8f6e-0820ee099da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:26:59.997542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-28402334-f35f-4fc9-8f6e-0820ee099da7 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:26:59.998731 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-28402334-f35f-4fc9-8f6e-0820ee099da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:00.006967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 832/1670] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:00.011239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17b13f-3764-4721-8461-2bafd95bd45d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:00.021178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17b13f-3764-4721-8461-2bafd95bd45d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:00.026466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:00.027084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:00.027363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:00.027636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:00.028308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:00.028622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:00.030234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:00.042795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 839/1671] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:27:00.047349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28402334-f35f-4fc9-8f6e-0820ee099da7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:00.057268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28402334-f35f-4fc9-8f6e-0820ee099da7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:00.061771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b148b9-0f81-43d2-ae38-4cf6ff2f4ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:00.062387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b148b9-0f81-43d2-ae38-4cf6ff2f4ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:00.062661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b148b9-0f81-43d2-ae38-4cf6ff2f4ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:00.062928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b148b9-0f81-43d2-ae38-4cf6ff2f4ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:00.063552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b148b9-0f81-43d2-ae38-4cf6ff2f4ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:00.065449 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b148b9-0f81-43d2-ae38-4cf6ff2f4ec6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:00.316246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 833/1672] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:00 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:00.316545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:00.325832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24dfc0-39c6-49f1-b88c-6683d714e126 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:00.382016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ebc69-af9d-44d5-b64d-d867ad4ee41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:00.382611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ebc69-af9d-44d5-b64d-d867ad4ee41d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:00.382886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ebc69-af9d-44d5-b64d-d867ad4ee41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:00.383160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ebc69-af9d-44d5-b64d-d867ad4ee41d admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:00.383775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ebc69-af9d-44d5-b64d-d867ad4ee41d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:00.385881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ebc69-af9d-44d5-b64d-d867ad4ee41d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:00.467307 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 840/1673] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:00 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:00.469231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b148b9-0f81-43d2-ae38-4cf6ff2f4ec6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:00.480800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b148b9-0f81-43d2-ae38-4cf6ff2f4ec6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:00.485399 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425ec04-2103-465d-8586-211ec4fd07f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:00.485994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425ec04-2103-465d-8586-211ec4fd07f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:00.486273 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425ec04-2103-465d-8586-211ec4fd07f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:00.486570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425ec04-2103-465d-8586-211ec4fd07f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:00.487212 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425ec04-2103-465d-8586-211ec4fd07f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:00.488829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425ec04-2103-465d-8586-211ec4fd07f8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:00.499047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 834/1674] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:00.502749 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ebc69-af9d-44d5-b64d-d867ad4ee41d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:00.512851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ebc69-af9d-44d5-b64d-d867ad4ee41d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:00.517744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60831927-1c9f-4865-b05a-b99b66ccecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:00.518420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60831927-1c9f-4865-b05a-b99b66ccecaa admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:00.518702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60831927-1c9f-4865-b05a-b99b66ccecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:00.518973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60831927-1c9f-4865-b05a-b99b66ccecaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:00.519644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60831927-1c9f-4865-b05a-b99b66ccecaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:00.521274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60831927-1c9f-4865-b05a-b99b66ccecaa admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:00.524663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:00.535129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 841/1675] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:00.539898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425ec04-2103-465d-8586-211ec4fd07f8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:00.549897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425ec04-2103-465d-8586-211ec4fd07f8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:00.564385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293addc-85a1-4f19-9f5c-5af661a8bb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:00.564385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293addc-85a1-4f19-9f5c-5af661a8bb33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:00.564385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293addc-85a1-4f19-9f5c-5af661a8bb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:00.564385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293addc-85a1-4f19-9f5c-5af661a8bb33 admin admin] PATH_INFO: `/v3/projects/ab0417eb6f1a41fa862f3ce62710fef5/users/a5cc8ba403cf4939bff23984a3e0689b/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:00.572155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293addc-85a1-4f19-9f5c-5af661a8bb33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0417eb6f1a41fa862f3ce62710fef5, user_id=a5cc8ba403cf4939bff23984a3e0689b, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:00.574837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293addc-85a1-4f19-9f5c-5af661a8bb33 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:00.646604 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 835/1676] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:00 2022] PUT /identity/v3/projects/ab0417eb6f1a41fa862f3ce62710fef5/users/a5cc8ba403cf4939bff23984a3e0689b/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:00.660148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756273a-d61b-4311-9827-ba4574d09518 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:00.660600 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756273a-d61b-4311-9827-ba4574d09518 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:00.660893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756273a-d61b-4311-9827-ba4574d09518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:00.698853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7756273a-d61b-4311-9827-ba4574d09518 None None] MFA Rules not processed for user `a5cc8ba403cf4939bff23984a3e0689b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:00.700186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7756273a-d61b-4311-9827-ba4574d09518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:00.780411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 842/1677] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:00.782040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293addc-85a1-4f19-9f5c-5af661a8bb33 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:00.846072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:00.857026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293addc-85a1-4f19-9f5c-5af661a8bb33 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:00.861696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c7d17-d5b6-4d02-abc1-f786227a9808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:00.866172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c7d17-d5b6-4d02-abc1-f786227a9808 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:00.866172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c7d17-d5b6-4d02-abc1-f786227a9808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:00.866172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c7d17-d5b6-4d02-abc1-f786227a9808 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:00.866172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c7d17-d5b6-4d02-abc1-f786227a9808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:00.866172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c7d17-d5b6-4d02-abc1-f786227a9808 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:00.955719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 836/1678] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:00 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:00.956895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756273a-d61b-4311-9827-ba4574d09518 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:00.969384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756273a-d61b-4311-9827-ba4574d09518 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:01.037093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2db25a-01a6-4cbc-8cf1-ad7760b13f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:01.037732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2db25a-01a6-4cbc-8cf1-ad7760b13f46 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.038010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2db25a-01a6-4cbc-8cf1-ad7760b13f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.038286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2db25a-01a6-4cbc-8cf1-ad7760b13f46 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.038963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2db25a-01a6-4cbc-8cf1-ad7760b13f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:01.040907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2db25a-01a6-4cbc-8cf1-ad7760b13f46 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:01.154128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 843/1679] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:00 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:01.154244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c7d17-d5b6-4d02-abc1-f786227a9808 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:01.159796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c7d17-d5b6-4d02-abc1-f786227a9808 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:01.164555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f27490-cea8-4b69-a2a6-2f32b11c6531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:01.165208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27490-cea8-4b69-a2a6-2f32b11c6531 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.165417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27490-cea8-4b69-a2a6-2f32b11c6531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.165685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27490-cea8-4b69-a2a6-2f32b11c6531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.180058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f27490-cea8-4b69-a2a6-2f32b11c6531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:01.180058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f27490-cea8-4b69-a2a6-2f32b11c6531 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:01.180058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 837/1680] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:01.188260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2db25a-01a6-4cbc-8cf1-ad7760b13f46 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:01.194904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2db25a-01a6-4cbc-8cf1-ad7760b13f46 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:01.199999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88031a69-bd2f-4ccf-872a-ca91dc868dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:01.200663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031a69-bd2f-4ccf-872a-ca91dc868dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.200933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031a69-bd2f-4ccf-872a-ca91dc868dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.201203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031a69-bd2f-4ccf-872a-ca91dc868dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.201832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88031a69-bd2f-4ccf-872a-ca91dc868dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:01.203392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88031a69-bd2f-4ccf-872a-ca91dc868dbb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:01.216442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 844/1681] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:01.219833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f27490-cea8-4b69-a2a6-2f32b11c6531 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:01.233975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f27490-cea8-4b69-a2a6-2f32b11c6531 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:01.239639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed5ba-18a7-4f53-b6e1-99455cdf7d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:01.240316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed5ba-18a7-4f53-b6e1-99455cdf7d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.241138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed5ba-18a7-4f53-b6e1-99455cdf7d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.242321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed5ba-18a7-4f53-b6e1-99455cdf7d7d admin admin] PATH_INFO: `/v3/projects/0373299f94fd46c5b1d9ae89b467131f/users/b7d8269d9bda4eb492cd945c71124cde/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.273063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ed5ba-18a7-4f53-b6e1-99455cdf7d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0373299f94fd46c5b1d9ae89b467131f, user_id=b7d8269d9bda4eb492cd945c71124cde, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:01.275910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ed5ba-18a7-4f53-b6e1-99455cdf7d7d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:01.373260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 838/1682] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:01 2022] PUT /identity/v3/projects/0373299f94fd46c5b1d9ae89b467131f/users/b7d8269d9bda4eb492cd945c71124cde/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:01.383534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47d127-7cb4-4402-8408-dd355915287a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.384036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47d127-7cb4-4402-8408-dd355915287a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.384135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47d127-7cb4-4402-8408-dd355915287a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.411705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5b47d127-7cb4-4402-8408-dd355915287a None None] MFA Rules not processed for user `b7d8269d9bda4eb492cd945c71124cde`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:01.415947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5b47d127-7cb4-4402-8408-dd355915287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:01.470158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:01.479546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 845/1683] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:01.484044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed5ba-18a7-4f53-b6e1-99455cdf7d7d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:01.545860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed5ba-18a7-4f53-b6e1-99455cdf7d7d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:01.550445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7f6f2-6005-4d62-8b4a-e1dce099af1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:01.551046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7f6f2-6005-4d62-8b4a-e1dce099af1f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.551324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7f6f2-6005-4d62-8b4a-e1dce099af1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.551595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7f6f2-6005-4d62-8b4a-e1dce099af1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.552213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7f6f2-6005-4d62-8b4a-e1dce099af1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:01.554742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7f6f2-6005-4d62-8b4a-e1dce099af1f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:01.635557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 839/1684] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:01 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:01.639401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47d127-7cb4-4402-8408-dd355915287a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:01.650216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47d127-7cb4-4402-8408-dd355915287a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:01.708127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e3e9e-d81c-4738-8892-edb9bfb924bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:01.708765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e3e9e-d81c-4738-8892-edb9bfb924bd admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.709062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e3e9e-d81c-4738-8892-edb9bfb924bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.709836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e3e9e-d81c-4738-8892-edb9bfb924bd admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.710492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e3e9e-d81c-4738-8892-edb9bfb924bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:01.712423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e3e9e-d81c-4738-8892-edb9bfb924bd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:01.817968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 846/1685] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:01 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:01.821940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7f6f2-6005-4d62-8b4a-e1dce099af1f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:01.832384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7f6f2-6005-4d62-8b4a-e1dce099af1f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:01.837676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794e5cf-a767-4721-bc07-58f9ca6288c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:01.838263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e5cf-a767-4721-bc07-58f9ca6288c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.838543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e5cf-a767-4721-bc07-58f9ca6288c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.838813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e5cf-a767-4721-bc07-58f9ca6288c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.839495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794e5cf-a767-4721-bc07-58f9ca6288c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:01.841118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794e5cf-a767-4721-bc07-58f9ca6288c5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:01.852127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 840/1686] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:01.856088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e3e9e-d81c-4738-8892-edb9bfb924bd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:01.866166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e3e9e-d81c-4738-8892-edb9bfb924bd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:01.870818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239098bc-1098-4316-99b9-96981a34b149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:01.871458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239098bc-1098-4316-99b9-96981a34b149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:01.871736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239098bc-1098-4316-99b9-96981a34b149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:01.872009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239098bc-1098-4316-99b9-96981a34b149 admin admin] PATH_INFO: `/v3/projects/43054f1a7fe5431899fb587c9df524f7/users/3b69d03af9e242efb81adde74f414b0a/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:01.893154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239098bc-1098-4316-99b9-96981a34b149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43054f1a7fe5431899fb587c9df524f7, user_id=3b69d03af9e242efb81adde74f414b0a, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:01.895753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239098bc-1098-4316-99b9-96981a34b149 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:01.968862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 847/1687] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:01 2022] PUT /identity/v3/projects/43054f1a7fe5431899fb587c9df524f7/users/3b69d03af9e242efb81adde74f414b0a/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:01.972994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794e5cf-a767-4721-bc07-58f9ca6288c5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:01.986982 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794e5cf-a767-4721-bc07-58f9ca6288c5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:02.048262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34885a2-13c6-4b66-99e7-df768b956aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:02.048890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34885a2-13c6-4b66-99e7-df768b956aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:02.049169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34885a2-13c6-4b66-99e7-df768b956aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:02.049445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34885a2-13c6-4b66-99e7-df768b956aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:02.050239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34885a2-13c6-4b66-99e7-df768b956aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:02.051796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34885a2-13c6-4b66-99e7-df768b956aaf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:02.061825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 841/1688] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:02.087472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239098bc-1098-4316-99b9-96981a34b149 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:02.100557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239098bc-1098-4316-99b9-96981a34b149 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:02.105407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:02.110123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:02.110123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:02.110123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] PATH_INFO: `/v3/projects/43054f1a7fe5431899fb587c9df524f7/users/3b69d03af9e242efb81adde74f414b0a/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:02.110123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43054f1a7fe5431899fb587c9df524f7, user_id=3b69d03af9e242efb81adde74f414b0a, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:02.111687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:02.122668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 848/1689] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:02 2022] PUT /identity/v3/projects/43054f1a7fe5431899fb587c9df524f7/users/3b69d03af9e242efb81adde74f414b0a/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:02.135082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba6d03-11a0-48cd-a794-65d0395bdf3a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:02.135369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba6d03-11a0-48cd-a794-65d0395bdf3a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:02.135642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba6d03-11a0-48cd-a794-65d0395bdf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:02.162356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:02.165214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-54ba6d03-11a0-48cd-a794-65d0395bdf3a None None] MFA Rules not processed for user `3b69d03af9e242efb81adde74f414b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:02.166390 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-54ba6d03-11a0-48cd-a794-65d0395bdf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:02.236981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 842/1690] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:02.259551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:02.261357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:02.321358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3e1f-a808-4062-b820-c92f4ce3f471 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:02.325024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc5d92-a2dd-4fe2-b02a-19e00dbe0f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:02.325688 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc5d92-a2dd-4fe2-b02a-19e00dbe0f35 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:02.325966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc5d92-a2dd-4fe2-b02a-19e00dbe0f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:02.326264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc5d92-a2dd-4fe2-b02a-19e00dbe0f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:02.382785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc5d92-a2dd-4fe2-b02a-19e00dbe0f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:02.384231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc5d92-a2dd-4fe2-b02a-19e00dbe0f35 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:02.396206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 849/1691] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:05.271751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba6d03-11a0-48cd-a794-65d0395bdf3a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:05.281667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba6d03-11a0-48cd-a794-65d0395bdf3a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:05.285365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:05.285970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:05.286254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:05.286528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:05.345898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:05.347331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:05.356786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 843/1692] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:08.246217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc5d92-a2dd-4fe2-b02a-19e00dbe0f35 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:08.250299 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:08.255557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc5d92-a2dd-4fe2-b02a-19e00dbe0f35 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:08.259665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817534a-ca3b-4c6d-96a1-31117f136411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:08.263740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817534a-ca3b-4c6d-96a1-31117f136411 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:08.263740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817534a-ca3b-4c6d-96a1-31117f136411 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:08.263740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817534a-ca3b-4c6d-96a1-31117f136411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:08.271074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817534a-ca3b-4c6d-96a1-31117f136411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:08.272531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817534a-ca3b-4c6d-96a1-31117f136411 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:08.282268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 850/1693] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:14.666044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:14.668034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:14.737052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1ce79-9867-43be-8fcf-2a01c6087547 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:14.741669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:14.742372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:14.742651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:14.742950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] PATH_INFO: `/v3/users/5153dd1df5524e7b88b4c81f22106255` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:14.744065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5153dd1df5524e7b88b4c81f22106255)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:14.746033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:14.861119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817534a-ca3b-4c6d-96a1-31117f136411 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:14.871854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817534a-ca3b-4c6d-96a1-31117f136411 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:14.876369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:14.877055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:14.877333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:14.877602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] PATH_INFO: `/v3/users/a5cc8ba403cf4939bff23984a3e0689b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:14.878741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5cc8ba403cf4939bff23984a3e0689b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:14.880683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:14.895265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:14.902133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5153dd1df5524e7b88b4c81f22106255'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:14.909568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5153dd1df5524e7b88b4c81f22106255'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:14.936762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5153dd1df5524e7b88b4c81f22106255'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:14.994743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5153dd1df5524e7b88b4c81f22106255'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:14.995374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-404fa24c-9e1d-4d59-935c-0f35d332d205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5153dd1df5524e7b88b4c81f22106255'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.001654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 844/1694] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:14 2022] DELETE /identity/v3/users/5153dd1df5524e7b88b4c81f22106255 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:15.005848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b0f08-064b-4e3f-925a-332490e58afe None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:15.006128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b0f08-064b-4e3f-925a-332490e58afe None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:15.006401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b0f08-064b-4e3f-925a-332490e58afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:15.025051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4d0b0f08-064b-4e3f-925a-332490e58afe None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:15.026269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4d0b0f08-064b-4e3f-925a-332490e58afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:15.100305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5cc8ba403cf4939bff23984a3e0689b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.100952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5cc8ba403cf4939bff23984a3e0689b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.105285 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 845/1695] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:15.112243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b0f08-064b-4e3f-925a-332490e58afe None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:15.113688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4d0b0f08-064b-4e3f-925a-332490e58afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:15.114751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5cc8ba403cf4939bff23984a3e0689b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.121770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5cc8ba403cf4939bff23984a3e0689b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.122524 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-21327b5b-0fbd-45fd-9f8b-0caa121c7041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5cc8ba403cf4939bff23984a3e0689b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.177798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b0f08-064b-4e3f-925a-332490e58afe None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:15.181311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:15.181908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:15.182176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:15.183033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:15.184369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:15.196185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 851/1696] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:14 2022] DELETE /identity/v3/users/a5cc8ba403cf4939bff23984a3e0689b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:15.200969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:15.201257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b094-5e94-4d50-8e5b-3dab0dd9477a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:15.201541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b094-5e94-4d50-8e5b-3dab0dd9477a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:15.201834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705b094-5e94-4d50-8e5b-3dab0dd9477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:15.202409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:15.213709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 846/1697] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:15.223939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0705b094-5e94-4d50-8e5b-3dab0dd9477a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:15.225443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0705b094-5e94-4d50-8e5b-3dab0dd9477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:15.286389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 852/1698] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:15.296923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:15.298928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:15.313753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7a873-3a81-426d-8b9e-50e577aaec31 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:15.364484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:15.365011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:15.365289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:15.365558 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:15.375314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:15.376754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:15.386229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 847/1699] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:15.684053 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705b094-5e94-4d50-8e5b-3dab0dd9477a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:15.685677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0705b094-5e94-4d50-8e5b-3dab0dd9477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:15.704565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705b094-5e94-4d50-8e5b-3dab0dd9477a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:15.718261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:15.719829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:15.720103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:15.720402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] PATH_INFO: `/v3/projects/388dfbe1d1f442df9fbd3dbdb5dd881b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:15.723372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=388dfbe1d1f442df9fbd3dbdb5dd881b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:15.725311 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:15.835097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '388dfbe1d1f442df9fbd3dbdb5dd881b', 'request_id': 'req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.881558 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:15.883102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:15.891900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:15.899562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '388dfbe1d1f442df9fbd3dbdb5dd881b', 'request_id': 'req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.900329 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579af69-da6c-40fe-8d33-3e56320a184b service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:15.902928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:15.909606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '388dfbe1d1f442df9fbd3dbdb5dd881b', 'request_id': 'req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.922849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] The token cache is being invalidate because project 388dfbe1d1f442df9fbd3dbdb5dd881b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:15.923152 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.924178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:15.925622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 853/1700] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:15 2022] DELETE /identity/v3/projects/388dfbe1d1f442df9fbd3dbdb5dd881b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:15.929529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:15.931515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:15.994295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:15.994942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:15.995256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:15.995535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] PATH_INFO: `/v3/projects/ab0417eb6f1a41fa862f3ce62710fef5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:15.997160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0417eb6f1a41fa862f3ce62710fef5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:16.000446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:16.040289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fe90f-d7ae-482d-bfbe-ce3c5100cf24 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:16.048471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:16.049157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:16.050033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:16.050033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] PATH_INFO: `/v3/users/4b85a04143fa4280861f79e42b0238cb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:16.050937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b85a04143fa4280861f79e42b0238cb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:16.052964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:16.117992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0417eb6f1a41fa862f3ce62710fef5', 'request_id': 'req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.124180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0417eb6f1a41fa862f3ce62710fef5', 'request_id': 'req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.130959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0417eb6f1a41fa862f3ce62710fef5', 'request_id': 'req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.193981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] The token cache is being invalidate because project ab0417eb6f1a41fa862f3ce62710fef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:16.194188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.194806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.196327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 848/1701] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:15 2022] DELETE /identity/v3/projects/ab0417eb6f1a41fa862f3ce62710fef5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:16.200882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:16.202374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:16.254674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b85a04143fa4280861f79e42b0238cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.255476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b85a04143fa4280861f79e42b0238cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.269227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b85a04143fa4280861f79e42b0238cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.270466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9c080-2ccd-4f73-9a0d-f94d5c71b55c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:16.275139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:16.275760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:16.277125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:16.277125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] PATH_INFO: `/v3/users/b7d8269d9bda4eb492cd945c71124cde` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:16.277315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b85a04143fa4280861f79e42b0238cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.277470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d8269d9bda4eb492cd945c71124cde)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:16.277610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b85a04143fa4280861f79e42b0238cb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.279342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:16.377271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 854/1702] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:15 2022] DELETE /identity/v3/users/4b85a04143fa4280861f79e42b0238cb => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:16.516112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d8269d9bda4eb492cd945c71124cde'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.520319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:16.525341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d8269d9bda4eb492cd945c71124cde'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.543288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d8269d9bda4eb492cd945c71124cde'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.594043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d8269d9bda4eb492cd945c71124cde'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.594674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d8269d9bda4eb492cd945c71124cde'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:16.601298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 849/1703] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:16 2022] DELETE /identity/v3/users/b7d8269d9bda4eb492cd945c71124cde => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:16.756382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:16.761570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:16.843744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc950f0-a00b-492d-8fa5-7ac886bd40d8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:16.856416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:16.922265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:16.922903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:16.923176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:16.923453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] PATH_INFO: `/v3/projects/a68cac5285214a728bd474d1c11a8ced` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:16.924911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68cac5285214a728bd474d1c11a8ced)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:16.926974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.064760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.066965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:17.081921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68cac5285214a728bd474d1c11a8ced', 'request_id': 'req-90a8979d-95f3-4171-a311-eaa861146b5d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.087718 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba3ae1-820a-4808-a042-df78834b6ef4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:17.142896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68cac5285214a728bd474d1c11a8ced', 'request_id': 'req-90a8979d-95f3-4171-a311-eaa861146b5d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.155213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68cac5285214a728bd474d1c11a8ced', 'request_id': 'req-90a8979d-95f3-4171-a311-eaa861146b5d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.172474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] The token cache is being invalidate because project a68cac5285214a728bd474d1c11a8ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:17.172767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.173429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-90a8979d-95f3-4171-a311-eaa861146b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.176033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 855/1704] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:16 2022] DELETE /identity/v3/projects/a68cac5285214a728bd474d1c11a8ced => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:17.180912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef324d-7b02-485a-bd26-fb475557921e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.181324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef324d-7b02-485a-bd26-fb475557921e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.181652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:17.181781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef324d-7b02-485a-bd26-fb475557921e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.182252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.182573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.184133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] PATH_INFO: `/v3/projects/0373299f94fd46c5b1d9ae89b467131f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.184133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0373299f94fd46c5b1d9ae89b467131f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:17.186176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.204651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a9ef324d-7b02-485a-bd26-fb475557921e None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:17.205984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a9ef324d-7b02-485a-bd26-fb475557921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:17.215316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 856/1705] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:17.223100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d5dab-1936-42dc-b2d2-77d8b04308df None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.223377 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d5dab-1936-42dc-b2d2-77d8b04308df None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.223649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d5dab-1936-42dc-b2d2-77d8b04308df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.244499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-426d5dab-1936-42dc-b2d2-77d8b04308df None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:17.245728 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-426d5dab-1936-42dc-b2d2-77d8b04308df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:17.254867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 857/1706] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:17.259190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d5dab-1936-42dc-b2d2-77d8b04308df None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.268986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d5dab-1936-42dc-b2d2-77d8b04308df None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:17.273889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:17.274485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.274767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.275040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.275682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:17.275992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:17.278139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.298264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 858/1707] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:27:17.300311 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.310253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59623af-e487-4ef4-b6a3-88f0e3acda0f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:17.311167 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0373299f94fd46c5b1d9ae89b467131f', 'request_id': 'req-af810ada-ffe3-40b9-9f82-2af7868ad73b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.317980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0373299f94fd46c5b1d9ae89b467131f', 'request_id': 'req-af810ada-ffe3-40b9-9f82-2af7868ad73b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.326281 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0373299f94fd46c5b1d9ae89b467131f', 'request_id': 'req-af810ada-ffe3-40b9-9f82-2af7868ad73b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.374470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:17.375104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.375382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.375753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.376385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:17.378297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.422215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] The token cache is being invalidate because project 0373299f94fd46c5b1d9ae89b467131f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:17.422503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.423120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.424624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 850/1708] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:17 2022] DELETE /identity/v3/projects/0373299f94fd46c5b1d9ae89b467131f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:17.428594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.429877 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:17.442021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810ada-ffe3-40b9-9f82-2af7868ad73b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:17.446745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:17.447346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.447624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.448757 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] PATH_INFO: `/v3/users/3b69d03af9e242efb81adde74f414b0a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.449568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b69d03af9e242efb81adde74f414b0a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:17.451507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.470097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:17.477439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 859/1709] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:17 2022] POST /identity/v3/projects => generated 401 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:17.481516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.483109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:17.546856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2ba99-1e1a-408b-80ef-0fec462e9c96 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:17.552450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dc19f-a424-412e-8947-0d3273ea9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:17.553050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dc19f-a424-412e-8947-0d3273ea9dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.553326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dc19f-a424-412e-8947-0d3273ea9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.553595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dc19f-a424-412e-8947-0d3273ea9dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.554213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dc19f-a424-412e-8947-0d3273ea9dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:17.556103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dc19f-a424-412e-8947-0d3273ea9dd9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.658013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:17.658262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 860/1710] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:17 2022] POST /identity/v3/users => generated 301 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:17.662466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dc19f-a424-412e-8947-0d3273ea9dd9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.667216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b69d03af9e242efb81adde74f414b0a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.674213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dc19f-a424-412e-8947-0d3273ea9dd9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:17.674857 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b69d03af9e242efb81adde74f414b0a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.679085 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ff730-7383-4b9b-93ff-fd1989912a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:17.679815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ff730-7383-4b9b-93ff-fd1989912a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.680163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ff730-7383-4b9b-93ff-fd1989912a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.680527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ff730-7383-4b9b-93ff-fd1989912a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.681183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ff730-7383-4b9b-93ff-fd1989912a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:17.683423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ff730-7383-4b9b-93ff-fd1989912a7a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.691213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b69d03af9e242efb81adde74f414b0a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.694595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 861/1711] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:17.699571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ff730-7383-4b9b-93ff-fd1989912a7a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.709624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ff730-7383-4b9b-93ff-fd1989912a7a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:17.714282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf176a8-55ec-4e06-a384-2f8e1aea827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:17.714851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf176a8-55ec-4e06-a384-2f8e1aea827d admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.715132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf176a8-55ec-4e06-a384-2f8e1aea827d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.715402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf176a8-55ec-4e06-a384-2f8e1aea827d admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.716015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf176a8-55ec-4e06-a384-2f8e1aea827d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:17.717620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf176a8-55ec-4e06-a384-2f8e1aea827d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.728062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 862/1712] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:17.732076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf176a8-55ec-4e06-a384-2f8e1aea827d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.741833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf176a8-55ec-4e06-a384-2f8e1aea827d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:17.742689 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b69d03af9e242efb81adde74f414b0a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.743302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-777f1e0e-1164-4f51-bb69-b4aff7d619c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b69d03af9e242efb81adde74f414b0a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:17.746920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:17.747542 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.747820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.748092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] PATH_INFO: `/v3/projects/dd0e49015850403ea4e718fb8602b9ff/users/3874dae87ec3415da49fe1aecf0f2982/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.750117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 851/1713] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:17 2022] DELETE /identity/v3/users/3b69d03af9e242efb81adde74f414b0a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:17.766608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0e49015850403ea4e718fb8602b9ff, user_id=3874dae87ec3415da49fe1aecf0f2982, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:17.769375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:17.845873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 863/1714] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:17 2022] PUT /identity/v3/projects/dd0e49015850403ea4e718fb8602b9ff/users/3874dae87ec3415da49fe1aecf0f2982/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:17.857322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd05d-9c4a-4eeb-8f98-bde538ac110d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:17.857604 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd05d-9c4a-4eeb-8f98-bde538ac110d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:17.857878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fd05d-9c4a-4eeb-8f98-bde538ac110d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:17.892030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-520fd05d-9c4a-4eeb-8f98-bde538ac110d None None] MFA Rules not processed for user `3874dae87ec3415da49fe1aecf0f2982`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:17.893340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-520fd05d-9c4a-4eeb-8f98-bde538ac110d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:17.974546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 852/1715] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:17 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:17.982768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:17.984638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:18.051386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f099-2d71-49e3-bb69-2ddc566ce27c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.056323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.056930 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.057210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.058010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.058652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.060563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.201707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 864/1716] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:17 2022] POST /identity/v3/projects => generated 399 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:18.206226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fd05d-9c4a-4eeb-8f98-bde538ac110d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.217218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fd05d-9c4a-4eeb-8f98-bde538ac110d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.225120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.227120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:18.271356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:18.282638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e489a-994c-49a6-b45c-49158ab1c80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.283246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e489a-994c-49a6-b45c-49158ab1c80a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.283522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e489a-994c-49a6-b45c-49158ab1c80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.283795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e489a-994c-49a6-b45c-49158ab1c80a admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.284451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e489a-994c-49a6-b45c-49158ab1c80a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.286370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e489a-994c-49a6-b45c-49158ab1c80a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.309317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b6cf1-4188-48a4-bdf1-cde1d1a8ee43 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.314267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.314895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.315171 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.315441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] PATH_INFO: `/v3/projects/43054f1a7fe5431899fb587c9df524f7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.316601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43054f1a7fe5431899fb587c9df524f7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.318594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.408023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 853/1717] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:18 2022] POST /identity/v3/users => generated 306 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:18.411769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e489a-994c-49a6-b45c-49158ab1c80a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.425117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e489a-994c-49a6-b45c-49158ab1c80a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.434511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df0f80-e583-4452-bdd9-89d0a3bec9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.434511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df0f80-e583-4452-bdd9-89d0a3bec9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.434511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df0f80-e583-4452-bdd9-89d0a3bec9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.434958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df0f80-e583-4452-bdd9-89d0a3bec9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.435584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df0f80-e583-4452-bdd9-89d0a3bec9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.437204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df0f80-e583-4452-bdd9-89d0a3bec9e0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.465741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 854/1718] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:18.467398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43054f1a7fe5431899fb587c9df524f7', 'request_id': 'req-fa11822a-071d-4803-8132-27bef041bf26'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:18.467734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df0f80-e583-4452-bdd9-89d0a3bec9e0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.475163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df0f80-e583-4452-bdd9-89d0a3bec9e0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.497503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:18.544756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.545417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.545692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.545961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] PATH_INFO: `/v3/projects/4ee9cd6caf244849ba550ecb7a734b55/users/4d7a001022aa426c9c88aca32135e818/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.550314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43054f1a7fe5431899fb587c9df524f7', 'request_id': 'req-fa11822a-071d-4803-8132-27bef041bf26'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:18.558286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43054f1a7fe5431899fb587c9df524f7', 'request_id': 'req-fa11822a-071d-4803-8132-27bef041bf26'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:18.561495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee9cd6caf244849ba550ecb7a734b55, user_id=4d7a001022aa426c9c88aca32135e818, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.564169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.572662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] The token cache is being invalidate because project 43054f1a7fe5431899fb587c9df524f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:18.572947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:18.573739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fa11822a-071d-4803-8132-27bef041bf26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:18.575359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 865/1719] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:18 2022] DELETE /identity/v3/projects/43054f1a7fe5431899fb587c9df524f7 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:18.580421 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da134-516f-47c8-838f-0cd27b9ac3f6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.590809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da134-516f-47c8-838f-0cd27b9ac3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.590809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da134-516f-47c8-838f-0cd27b9ac3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.601324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-6c5da134-516f-47c8-838f-0cd27b9ac3f6 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:18.602564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6c5da134-516f-47c8-838f-0cd27b9ac3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:18.611722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 866/1720] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:18.620779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97324974-1e8b-4db9-87dc-1802860427ba None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.621144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97324974-1e8b-4db9-87dc-1802860427ba None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.621489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97324974-1e8b-4db9-87dc-1802860427ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.643662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 855/1721] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:18 2022] PUT /identity/v3/projects/4ee9cd6caf244849ba550ecb7a734b55/users/4d7a001022aa426c9c88aca32135e818/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:18.644681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-97324974-1e8b-4db9-87dc-1802860427ba None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:18.645072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.646121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97324974-1e8b-4db9-87dc-1802860427ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:18.647182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:18.717963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 867/1722] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:18.722240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97324974-1e8b-4db9-87dc-1802860427ba None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.727769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777424b-bffb-471e-ade8-e95e7b14bf6d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.732378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97324974-1e8b-4db9-87dc-1802860427ba None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.733273 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65459e08-b31f-46db-8fe1-215f03de2c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.733892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65459e08-b31f-46db-8fe1-215f03de2c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.739288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65459e08-b31f-46db-8fe1-215f03de2c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.739665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65459e08-b31f-46db-8fe1-215f03de2c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.740389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65459e08-b31f-46db-8fe1-215f03de2c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.741069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.741749 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.742018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65459e08-b31f-46db-8fe1-215f03de2c27 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.742147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.742460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.743181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:18.743553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.745337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.755801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 856/1723] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:18.758323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 868/1724] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:27:18.759720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65459e08-b31f-46db-8fe1-215f03de2c27 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.762850 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.772802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65459e08-b31f-46db-8fe1-215f03de2c27 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.774292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2527118-643f-4994-a2cc-7bf7aedf0de4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.778925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a45c5b-3c1a-4015-b129-5bea2f1d690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.779604 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a45c5b-3c1a-4015-b129-5bea2f1d690a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.779887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a45c5b-3c1a-4015-b129-5bea2f1d690a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.780135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ccf90-c885-4c6f-ae06-b0ca678a2fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.780290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a45c5b-3c1a-4015-b129-5bea2f1d690a admin admin] PATH_INFO: `/v3/projects/4ee9cd6caf244849ba550ecb7a734b55/users/4d7a001022aa426c9c88aca32135e818/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.781022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ccf90-c885-4c6f-ae06-b0ca678a2fae admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.781327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ccf90-c885-4c6f-ae06-b0ca678a2fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.781687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ccf90-c885-4c6f-ae06-b0ca678a2fae admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.782350 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a45c5b-3c1a-4015-b129-5bea2f1d690a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee9cd6caf244849ba550ecb7a734b55, user_id=4d7a001022aa426c9c88aca32135e818, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.782473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ccf90-c885-4c6f-ae06-b0ca678a2fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:18.784541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ccf90-c885-4c6f-ae06-b0ca678a2fae admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.785207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a45c5b-3c1a-4015-b129-5bea2f1d690a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:18.798444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 857/1725] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:18 2022] PUT /identity/v3/projects/4ee9cd6caf244849ba550ecb7a734b55/users/4d7a001022aa426c9c88aca32135e818/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:18.809961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832abd8d-6015-4513-b883-f52a0dcab2cf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.810764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832abd8d-6015-4513-b883-f52a0dcab2cf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.811099 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832abd8d-6015-4513-b883-f52a0dcab2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.844687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-832abd8d-6015-4513-b883-f52a0dcab2cf None None] MFA Rules not processed for user `4d7a001022aa426c9c88aca32135e818`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:18.845956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-832abd8d-6015-4513-b883-f52a0dcab2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:18.887213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 869/1726] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:18 2022] POST /identity/v3/projects => generated 391 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:18.893104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ccf90-c885-4c6f-ae06-b0ca678a2fae admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.915823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:18.941631 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 858/1727] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:18.957005 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832abd8d-6015-4513-b883-f52a0dcab2cf None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:18.958461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-832abd8d-6015-4513-b883-f52a0dcab2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:18.988622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ccf90-c885-4c6f-ae06-b0ca678a2fae admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:18.994823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41975845-ea8d-4f6d-8a1e-22b8111e378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:18.996304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41975845-ea8d-4f6d-8a1e-22b8111e378c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:18.996845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41975845-ea8d-4f6d-8a1e-22b8111e378c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:18.997174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41975845-ea8d-4f6d-8a1e-22b8111e378c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:18.998030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:18.998621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41975845-ea8d-4f6d-8a1e-22b8111e378c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:19.000965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41975845-ea8d-4f6d-8a1e-22b8111e378c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:19.046279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832abd8d-6015-4513-b883-f52a0dcab2cf None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:19.101687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ad5b-a57d-477d-a742-b4591edb8a60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:19.102233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ad5b-a57d-477d-a742-b4591edb8a60 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:19.103133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ad5b-a57d-477d-a742-b4591edb8a60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:19.103133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ad5b-a57d-477d-a742-b4591edb8a60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:19.108302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 870/1728] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:18 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:19.112364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41975845-ea8d-4f6d-8a1e-22b8111e378c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:19.123414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41975845-ea8d-4f6d-8a1e-22b8111e378c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:19.128672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465d4c2-81ff-4343-8b9b-440ae3026668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:19.129130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465d4c2-81ff-4343-8b9b-440ae3026668 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:19.129404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465d4c2-81ff-4343-8b9b-440ae3026668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:19.129671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465d4c2-81ff-4343-8b9b-440ae3026668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:19.130506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465d4c2-81ff-4343-8b9b-440ae3026668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:19.136283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465d4c2-81ff-4343-8b9b-440ae3026668 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:19.143705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 871/1729] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:19.147506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465d4c2-81ff-4343-8b9b-440ae3026668 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:19.159706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465d4c2-81ff-4343-8b9b-440ae3026668 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:19.165545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ff7d0-e593-4f1e-866c-3c61ec9a204a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:19.166323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ff7d0-e593-4f1e-866c-3c61ec9a204a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:19.166688 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ff7d0-e593-4f1e-866c-3c61ec9a204a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:19.167067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ff7d0-e593-4f1e-866c-3c61ec9a204a admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:19.167787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ff7d0-e593-4f1e-866c-3c61ec9a204a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:19.169472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ff7d0-e593-4f1e-866c-3c61ec9a204a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:19.172449 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ad5b-a57d-477d-a742-b4591edb8a60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:19.173796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ad5b-a57d-477d-a742-b4591edb8a60 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:19.183379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 872/1730] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:19.187290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ff7d0-e593-4f1e-866c-3c61ec9a204a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:19.187969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 859/1731] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:19.201839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ff7d0-e593-4f1e-866c-3c61ec9a204a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:19.213241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b7fbc-c521-4760-ad30-56434712fbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:19.214020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b7fbc-c521-4760-ad30-56434712fbd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:19.214359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b7fbc-c521-4760-ad30-56434712fbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:19.214699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b7fbc-c521-4760-ad30-56434712fbd3 admin admin] PATH_INFO: `/v3/projects/058eabdc497c4ab9abbb31f74656ae71/users/cf1420e3bb664fc0b0df46984fde385c/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:19.235612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b7fbc-c521-4760-ad30-56434712fbd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058eabdc497c4ab9abbb31f74656ae71, user_id=cf1420e3bb664fc0b0df46984fde385c, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:19.238280 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b7fbc-c521-4760-ad30-56434712fbd3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:19.320507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 873/1732] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:19 2022] PUT /identity/v3/projects/058eabdc497c4ab9abbb31f74656ae71/users/cf1420e3bb664fc0b0df46984fde385c/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:19.332542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ff98-b807-4739-bfe1-3a29b31d1f20 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:19.333020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ff98-b807-4739-bfe1-3a29b31d1f20 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:19.333110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ff98-b807-4739-bfe1-3a29b31d1f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:19.367045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d582ff98-b807-4739-bfe1-3a29b31d1f20 None None] MFA Rules not processed for user `cf1420e3bb664fc0b0df46984fde385c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:19.368329 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d582ff98-b807-4739-bfe1-3a29b31d1f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:19.383633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b7fbc-c521-4760-ad30-56434712fbd3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:19.440119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 860/1733] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:19.453168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582ff98-b807-4739-bfe1-3a29b31d1f20 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:19.453463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b7fbc-c521-4760-ad30-56434712fbd3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:19.454921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d582ff98-b807-4739-bfe1-3a29b31d1f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:19.457561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fb8ad-dbbf-4d0b-8ee5-5c4210151ee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:19.458265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fb8ad-dbbf-4d0b-8ee5-5c4210151ee1 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:19.458661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fb8ad-dbbf-4d0b-8ee5-5c4210151ee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:19.459026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fb8ad-dbbf-4d0b-8ee5-5c4210151ee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:19.460839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e62fb8ad-dbbf-4d0b-8ee5-5c4210151ee1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:19.470570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582ff98-b807-4739-bfe1-3a29b31d1f20 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:19.476863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdebfb3-c500-44f4-b6f5-ca58860c0b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:19.477650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdebfb3-c500-44f4-b6f5-ca58860c0b5f service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:19.478254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdebfb3-c500-44f4-b6f5-ca58860c0b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:19.478537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdebfb3-c500-44f4-b6f5-ca58860c0b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:19.486507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:19.489224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdebfb3-c500-44f4-b6f5-ca58860c0b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:19.491488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdebfb3-c500-44f4-b6f5-ca58860c0b5f service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:19.502024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 861/1734] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:19.533161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fb8ad-dbbf-4d0b-8ee5-5c4210151ee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:19.534944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fb8ad-dbbf-4d0b-8ee5-5c4210151ee1 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:19.546590 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 874/1735] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:19 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:23.025787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b369-0aca-4cc1-895a-4a50918926a7 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:23.026613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b369-0aca-4cc1-895a-4a50918926a7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:23.026894 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b369-0aca-4cc1-895a-4a50918926a7 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:23.028463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 862/1736] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:27:23.055812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d9708-c69b-47c1-a86b-3694b6eaf930 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:23.056327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d9708-c69b-47c1-a86b-3694b6eaf930 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:23.062503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d9708-c69b-47c1-a86b-3694b6eaf930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:23.077798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0e0d9708-c69b-47c1-a86b-3694b6eaf930 None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:23.079198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0e0d9708-c69b-47c1-a86b-3694b6eaf930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:23.088915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:23.146503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 875/1737] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:27:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:23.159719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd620a56-3cfe-440f-8148-80140f8ea4bb None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:23.160078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd620a56-3cfe-440f-8148-80140f8ea4bb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:23.160493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd620a56-3cfe-440f-8148-80140f8ea4bb None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:23.162336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 863/1738] 10.222.0.48 () {54 vars in 961 bytes} [Mon Oct 24 13:27:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:27:23.169011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6700d-8d6c-49d9-a273-3652eb23be7a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:23.169011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6700d-8d6c-49d9-a273-3652eb23be7a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:23.175621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6700d-8d6c-49d9-a273-3652eb23be7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:23.191197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fad6700d-8d6c-49d9-a273-3652eb23be7a None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:23.191836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd620a56-3cfe-440f-8148-80140f8ea4bb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:23.192448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fad6700d-8d6c-49d9-a273-3652eb23be7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:23.194082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd620a56-3cfe-440f-8148-80140f8ea4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:23.202519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 876/1739] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Oct 24 13:27:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:23.210232 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6700d-8d6c-49d9-a273-3652eb23be7a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:23.219694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6700d-8d6c-49d9-a273-3652eb23be7a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:23.226984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b039273-b6da-4821-8d2e-d79e7c983fe2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:23.227825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039273-b6da-4821-8d2e-d79e7c983fe2 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:23.228111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039273-b6da-4821-8d2e-d79e7c983fe2 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:23.228413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b039273-b6da-4821-8d2e-d79e7c983fe2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:23.237489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b039273-b6da-4821-8d2e-d79e7c983fe2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:23.239051 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b039273-b6da-4821-8d2e-d79e7c983fe2 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:23.249655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 877/1740] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:23.264630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd620a56-3cfe-440f-8148-80140f8ea4bb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:23.270903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:23.271557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:23.271837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:23.272106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] PATH_INFO: `/v3/users/495ed7e994004db99f1553ab6e0cc65c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:23.273448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=495ed7e994004db99f1553ab6e0cc65c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:23.275380 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:23.432083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '495ed7e994004db99f1553ab6e0cc65c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:23.442170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '495ed7e994004db99f1553ab6e0cc65c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:23.454774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '495ed7e994004db99f1553ab6e0cc65c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:23.514136 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '495ed7e994004db99f1553ab6e0cc65c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:23.514771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-13438c79-f6d6-4e20-8e59-f935977bac89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '495ed7e994004db99f1553ab6e0cc65c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:23.522936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 864/1741] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:23 2022] DELETE /identity/v3/users/495ed7e994004db99f1553ab6e0cc65c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:23.763368 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b039273-b6da-4821-8d2e-d79e7c983fe2 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:23.775897 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b039273-b6da-4821-8d2e-d79e7c983fe2 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:23.838679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:23.839840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:23.840119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:23.840424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] PATH_INFO: `/v3/projects/6019625869c249fbae42eff10c82c590` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:23.841523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6019625869c249fbae42eff10c82c590)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:23.843419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:23.954438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6019625869c249fbae42eff10c82c590', 'request_id': 'req-cc9ea796-559a-4ae4-9a1b-30090323e6b4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:24.016444 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6019625869c249fbae42eff10c82c590', 'request_id': 'req-cc9ea796-559a-4ae4-9a1b-30090323e6b4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:24.023305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6019625869c249fbae42eff10c82c590', 'request_id': 'req-cc9ea796-559a-4ae4-9a1b-30090323e6b4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:24.034895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] The token cache is being invalidate because project 6019625869c249fbae42eff10c82c590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:24.035183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:24.035836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc9ea796-559a-4ae4-9a1b-30090323e6b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:24.037340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 878/1742] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:23 2022] DELETE /identity/v3/projects/6019625869c249fbae42eff10c82c590 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:24.053617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f535f-163c-4660-aabd-6a6b0d493995 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.054152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f535f-163c-4660-aabd-6a6b0d493995 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.054152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f535f-163c-4660-aabd-6a6b0d493995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.074178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-859f535f-163c-4660-aabd-6a6b0d493995 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:24.075361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-859f535f-163c-4660-aabd-6a6b0d493995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:24.150642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 865/1743] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:24.159159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef7b8-5e3e-4227-b3a0-ffae0a95d30c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.159652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef7b8-5e3e-4227-b3a0-ffae0a95d30c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.159760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef7b8-5e3e-4227-b3a0-ffae0a95d30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.180694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-38cef7b8-5e3e-4227-b3a0-ffae0a95d30c None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:24.181389 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-38cef7b8-5e3e-4227-b3a0-ffae0a95d30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:24.191080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 879/1744] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:24.195108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f535f-163c-4660-aabd-6a6b0d493995 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:24.206064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f535f-163c-4660-aabd-6a6b0d493995 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:24.211050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:24.211654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.211935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.212207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.212879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:24.213181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:24.215318 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:24.228551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 866/1745] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:27:24.232867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cef7b8-5e3e-4227-b3a0-ffae0a95d30c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:24.242304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cef7b8-5e3e-4227-b3a0-ffae0a95d30c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:24.247022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a823-2381-4c90-93ee-b5e477a7d68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:24.247604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a823-2381-4c90-93ee-b5e477a7d68e admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.252723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a823-2381-4c90-93ee-b5e477a7d68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.252723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a823-2381-4c90-93ee-b5e477a7d68e admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.252723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00a823-2381-4c90-93ee-b5e477a7d68e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:24.252723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00a823-2381-4c90-93ee-b5e477a7d68e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:24.265704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:24.359804 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 880/1746] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:24 2022] POST /identity/v3/projects => generated 405 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:24.361511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:24.366693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:24.372581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92252ec-c361-4b3f-b32b-729184162e8e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:24.428621 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8ade7-fea5-482f-83c8-d03f4ab3df14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:24.429224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ade7-fea5-482f-83c8-d03f4ab3df14 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.430671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ade7-fea5-482f-83c8-d03f4ab3df14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.430671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ade7-fea5-482f-83c8-d03f4ab3df14 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.430671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8ade7-fea5-482f-83c8-d03f4ab3df14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:24.432411 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8ade7-fea5-482f-83c8-d03f4ab3df14 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:24.514515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 867/1747] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:24 2022] POST /identity/v3/users => generated 303 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:24.518548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a823-2381-4c90-93ee-b5e477a7d68e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:24.530232 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a823-2381-4c90-93ee-b5e477a7d68e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:24.535057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7321d-c579-48cc-ac92-81299a710acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:24.535663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7321d-c579-48cc-ac92-81299a710acc admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.535941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7321d-c579-48cc-ac92-81299a710acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.536210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7321d-c579-48cc-ac92-81299a710acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.536891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7321d-c579-48cc-ac92-81299a710acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:24.538615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7321d-c579-48cc-ac92-81299a710acc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:24.550508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 881/1748] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:24.555640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8ade7-fea5-482f-83c8-d03f4ab3df14 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:24.579459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8ade7-fea5-482f-83c8-d03f4ab3df14 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:24.584395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acff2d6-19c3-46e7-92e0-738463d3e818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:24.585193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acff2d6-19c3-46e7-92e0-738463d3e818 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.585475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acff2d6-19c3-46e7-92e0-738463d3e818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.585750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acff2d6-19c3-46e7-92e0-738463d3e818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.586382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acff2d6-19c3-46e7-92e0-738463d3e818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:24.587995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acff2d6-19c3-46e7-92e0-738463d3e818 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:24.602503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 868/1749] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:24.603739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7321d-c579-48cc-ac92-81299a710acc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:24.614203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7321d-c579-48cc-ac92-81299a710acc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:24.618972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:24.619561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.619842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.620116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] PATH_INFO: `/v3/projects/acdf013b7f44452f90a21e22432b0015/users/f69a44bf981a495c91738c9172316a7f/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.635674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acdf013b7f44452f90a21e22432b0015, user_id=f69a44bf981a495c91738c9172316a7f, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:24.638387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:24.680645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 882/1750] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:24 2022] PUT /identity/v3/projects/acdf013b7f44452f90a21e22432b0015/users/f69a44bf981a495c91738c9172316a7f/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:24.685495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54567065-8f05-46f6-a77b-87a03b03d8c2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.686075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54567065-8f05-46f6-a77b-87a03b03d8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.686413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54567065-8f05-46f6-a77b-87a03b03d8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.707431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-54567065-8f05-46f6-a77b-87a03b03d8c2 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:24.708741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-54567065-8f05-46f6-a77b-87a03b03d8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:24.766550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 869/1751] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:24.775931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:24.778141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:24.834562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39f7f6-59e9-4239-b9c5-580cbf654c85 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:24.838166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602ddca-3b4d-49f8-826f-5c2b266d04a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:24.838759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602ddca-3b4d-49f8-826f-5c2b266d04a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:24.839628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602ddca-3b4d-49f8-826f-5c2b266d04a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:24.839912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602ddca-3b4d-49f8-826f-5c2b266d04a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:24.848730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602ddca-3b4d-49f8-826f-5c2b266d04a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:24.850136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602ddca-3b4d-49f8-826f-5c2b266d04a4 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:24.859367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 883/1752] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:25.996110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9509e-e0b7-4c48-928a-559ebd36c129 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:26.001163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9509e-e0b7-4c48-928a-559ebd36c129 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:26.001163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9509e-e0b7-4c48-928a-559ebd36c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:26.023575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f3b9509e-e0b7-4c48-928a-559ebd36c129 None None] MFA Rules not processed for user `f69a44bf981a495c91738c9172316a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:26.024806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f3b9509e-e0b7-4c48-928a-559ebd36c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:26.099029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 870/1753] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:25 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:26.103607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602ddca-3b4d-49f8-826f-5c2b266d04a4 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:26.113732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602ddca-3b4d-49f8-826f-5c2b266d04a4 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:26.118215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b3afe-ac61-4169-ba1e-546abf82a016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:26.118797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b3afe-ac61-4169-ba1e-546abf82a016 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:26.119076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b3afe-ac61-4169-ba1e-546abf82a016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:26.119342 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b3afe-ac61-4169-ba1e-546abf82a016 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:26.119970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b3afe-ac61-4169-ba1e-546abf82a016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:26.121925 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b3afe-ac61-4169-ba1e-546abf82a016 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:26.194471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 884/1754] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:26 2022] POST /identity/v3/projects => generated 403 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:26.198598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9509e-e0b7-4c48-928a-559ebd36c129 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:26.209351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9509e-e0b7-4c48-928a-559ebd36c129 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:26.217414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:26.264898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf06e-69b3-473c-96bc-0f24ab008096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:26.265497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf06e-69b3-473c-96bc-0f24ab008096 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:26.266344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf06e-69b3-473c-96bc-0f24ab008096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:26.266457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaf06e-69b3-473c-96bc-0f24ab008096 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:26.267114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfaf06e-69b3-473c-96bc-0f24ab008096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:26.269117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfaf06e-69b3-473c-96bc-0f24ab008096 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:26.371524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 871/1755] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:26 2022] POST /identity/v3/users => generated 308 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:26.375603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b3afe-ac61-4169-ba1e-546abf82a016 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:26.385758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b3afe-ac61-4169-ba1e-546abf82a016 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:26.390781 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6fcdd-9282-45c8-8485-414144a70257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:26.391360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6fcdd-9282-45c8-8485-414144a70257 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:26.391633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6fcdd-9282-45c8-8485-414144a70257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:26.391940 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6fcdd-9282-45c8-8485-414144a70257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:26.392592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6fcdd-9282-45c8-8485-414144a70257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:26.394166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6fcdd-9282-45c8-8485-414144a70257 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:26.405082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 885/1756] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:26.409255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf06e-69b3-473c-96bc-0f24ab008096 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:26.419024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaf06e-69b3-473c-96bc-0f24ab008096 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:26.423575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ee5f2-c168-4519-b0f8-4f06a15b789b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:26.424207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee5f2-c168-4519-b0f8-4f06a15b789b admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:26.424520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee5f2-c168-4519-b0f8-4f06a15b789b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:26.424816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ee5f2-c168-4519-b0f8-4f06a15b789b admin admin] PATH_INFO: `/v3/projects/5b6c8a973f0d4c04b4010302750f72c6/users/9c841004523a4d6cbbdab8f298d4046f/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:26.439359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ee5f2-c168-4519-b0f8-4f06a15b789b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6c8a973f0d4c04b4010302750f72c6, user_id=9c841004523a4d6cbbdab8f298d4046f, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:26.442064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ee5f2-c168-4519-b0f8-4f06a15b789b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:26.518944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 872/1757] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:26 2022] PUT /identity/v3/projects/5b6c8a973f0d4c04b4010302750f72c6/users/9c841004523a4d6cbbdab8f298d4046f/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:26.522807 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6fcdd-9282-45c8-8485-414144a70257 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:26.533274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6fcdd-9282-45c8-8485-414144a70257 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:26.588481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063f467-df6c-444c-88eb-56a5899db681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:26.589112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063f467-df6c-444c-88eb-56a5899db681 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:26.589390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063f467-df6c-444c-88eb-56a5899db681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:26.589661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063f467-df6c-444c-88eb-56a5899db681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:26.590873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063f467-df6c-444c-88eb-56a5899db681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:26.592506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063f467-df6c-444c-88eb-56a5899db681 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:26.602388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 886/1758] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:26.606515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ee5f2-c168-4519-b0f8-4f06a15b789b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:26.620512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ee5f2-c168-4519-b0f8-4f06a15b789b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:26.625214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86e4ec-27bf-420e-be05-4429cda9bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:26.625864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86e4ec-27bf-420e-be05-4429cda9bf77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:26.626175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86e4ec-27bf-420e-be05-4429cda9bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:26.626449 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86e4ec-27bf-420e-be05-4429cda9bf77 admin admin] PATH_INFO: `/v3/projects/5b6c8a973f0d4c04b4010302750f72c6/users/9c841004523a4d6cbbdab8f298d4046f/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:26.628353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86e4ec-27bf-420e-be05-4429cda9bf77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6c8a973f0d4c04b4010302750f72c6, user_id=9c841004523a4d6cbbdab8f298d4046f, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:26.630966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86e4ec-27bf-420e-be05-4429cda9bf77 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:26.641876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 873/1759] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:26 2022] PUT /identity/v3/projects/5b6c8a973f0d4c04b4010302750f72c6/users/9c841004523a4d6cbbdab8f298d4046f/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:27.829107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996f79-527b-4da3-a88a-c92ce6fc8a0d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:27.841964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996f79-527b-4da3-a88a-c92ce6fc8a0d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:27.841964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18996f79-527b-4da3-a88a-c92ce6fc8a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:27.860074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-18996f79-527b-4da3-a88a-c92ce6fc8a0d None None] MFA Rules not processed for user `9c841004523a4d6cbbdab8f298d4046f`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:27.861352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-18996f79-527b-4da3-a88a-c92ce6fc8a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:27.874293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:27.936012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 887/1760] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:28.540297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b80f1e-fa53-4f90-b46a-865d355efe48 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:28.540795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b80f1e-fa53-4f90-b46a-865d355efe48 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:28.540889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b80f1e-fa53-4f90-b46a-865d355efe48 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:28.542226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 874/1761] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:27:28.554047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97923a8b-f01a-43c1-861e-0829075365bf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:28.554404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97923a8b-f01a-43c1-861e-0829075365bf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:28.554744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97923a8b-f01a-43c1-861e-0829075365bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:28.574984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-97923a8b-f01a-43c1-861e-0829075365bf None None] MFA Rules not processed for user `8dd54892919e41f9b1f2ad734dc554f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:28.576160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97923a8b-f01a-43c1-861e-0829075365bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:28.640634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 888/1762] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:28.651374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b80f1e-fa53-4f90-b46a-865d355efe48 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:28.653169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-70b80f1e-fa53-4f90-b46a-865d355efe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:28.670506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b80f1e-fa53-4f90-b46a-865d355efe48 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:28.677242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098945a-20d5-4ab8-9152-7c88497ba5c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:28.677242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098945a-20d5-4ab8-9152-7c88497ba5c4 service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:28.677242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098945a-20d5-4ab8-9152-7c88497ba5c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:28.677242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098945a-20d5-4ab8-9152-7c88497ba5c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:28.686348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098945a-20d5-4ab8-9152-7c88497ba5c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:28.687904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098945a-20d5-4ab8-9152-7c88497ba5c4 service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:28.719564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 875/1763] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:29.818334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97923a8b-f01a-43c1-861e-0829075365bf None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:29.889245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97923a8b-f01a-43c1-861e-0829075365bf None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:29.893790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:29.894413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:29.894695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:29.894966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] PATH_INFO: `/v3/users/f69a44bf981a495c91738c9172316a7f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:29.896102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69a44bf981a495c91738c9172316a7f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:29.898080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:30.116203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69a44bf981a495c91738c9172316a7f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.116894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69a44bf981a495c91738c9172316a7f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.118319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:30.129200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69a44bf981a495c91738c9172316a7f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.136026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69a44bf981a495c91738c9172316a7f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.137190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69a44bf981a495c91738c9172316a7f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.228503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 889/1764] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:29 2022] DELETE /identity/v3/users/f69a44bf981a495c91738c9172316a7f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:30.475131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098945a-20d5-4ab8-9152-7c88497ba5c4 service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:30.485824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098945a-20d5-4ab8-9152-7c88497ba5c4 service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:30.543303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:30.543931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:30.544224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:30.544546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] PATH_INFO: `/v3/projects/acdf013b7f44452f90a21e22432b0015` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:30.545670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=acdf013b7f44452f90a21e22432b0015)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:30.547607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:30.650991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acdf013b7f44452f90a21e22432b0015', 'request_id': 'req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.652918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:30.658074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acdf013b7f44452f90a21e22432b0015', 'request_id': 'req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.664715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acdf013b7f44452f90a21e22432b0015', 'request_id': 'req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.720937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] The token cache is being invalidate because project acdf013b7f44452f90a21e22432b0015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:30.721984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.722897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.724935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 876/1765] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:30 2022] DELETE /identity/v3/projects/acdf013b7f44452f90a21e22432b0015 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:30.728827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:30.731404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:30.747215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7c706-e751-4cff-9741-de0985df182b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:30.809246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:30.809884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:30.810162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:30.810445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] PATH_INFO: `/v3/users/9c841004523a4d6cbbdab8f298d4046f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:30.811539 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c841004523a4d6cbbdab8f298d4046f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:30.813511 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:30.992101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c841004523a4d6cbbdab8f298d4046f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:30.992925 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c841004523a4d6cbbdab8f298d4046f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.008774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c841004523a4d6cbbdab8f298d4046f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.017620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c841004523a4d6cbbdab8f298d4046f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.019755 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c841004523a4d6cbbdab8f298d4046f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.073898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 890/1766] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:30 2022] DELETE /identity/v3/users/9c841004523a4d6cbbdab8f298d4046f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:31.112431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:31.120428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:31.126987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f82ecf-16f7-4d19-95c3-44d5b52072ab admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:31.186863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:31.187513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:31.187793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:31.188070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] PATH_INFO: `/v3/users/3874dae87ec3415da49fe1aecf0f2982` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:31.189240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3874dae87ec3415da49fe1aecf0f2982)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:31.191155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:31.355910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:31.357376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:31.372853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad108ecc-c61e-49d5-84cd-0fd5c4033164 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:31.378369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:31.379104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:31.379330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:31.379598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] PATH_INFO: `/v3/projects/5b6c8a973f0d4c04b4010302750f72c6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:31.387619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b6c8a973f0d4c04b4010302750f72c6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:31.387619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:31.419796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3874dae87ec3415da49fe1aecf0f2982'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.426826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3874dae87ec3415da49fe1aecf0f2982'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.439584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3874dae87ec3415da49fe1aecf0f2982'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.484199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:31.490823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b6c8a973f0d4c04b4010302750f72c6', 'request_id': 'req-625ce616-04b2-444e-8f74-8b84a336c8e5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.496935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3874dae87ec3415da49fe1aecf0f2982'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.497533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0e240918-361b-450f-8571-6fb277a544cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3874dae87ec3415da49fe1aecf0f2982'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.503723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 877/1767] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:31 2022] DELETE /identity/v3/users/3874dae87ec3415da49fe1aecf0f2982 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:31.508156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a505ea-0149-4426-8ef0-b3204a5904bf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:31.508492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a505ea-0149-4426-8ef0-b3204a5904bf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:31.508772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a505ea-0149-4426-8ef0-b3204a5904bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:31.528125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-32a505ea-0149-4426-8ef0-b3204a5904bf None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:31.529355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-32a505ea-0149-4426-8ef0-b3204a5904bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:31.583867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:31.592438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 878/1768] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:31.602068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a505ea-0149-4426-8ef0-b3204a5904bf None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:31.603393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-32a505ea-0149-4426-8ef0-b3204a5904bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:31.658218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a505ea-0149-4426-8ef0-b3204a5904bf None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:31.661819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b526fb-a9ae-474e-bad0-bcb0fa656cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:31.662422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b526fb-a9ae-474e-bad0-bcb0fa656cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:31.662694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b526fb-a9ae-474e-bad0-bcb0fa656cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:31.662963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b526fb-a9ae-474e-bad0-bcb0fa656cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:31.673637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b526fb-a9ae-474e-bad0-bcb0fa656cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:31.675780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b526fb-a9ae-474e-bad0-bcb0fa656cd4 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:31.685960 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 879/1769] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:31.880035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b6c8a973f0d4c04b4010302750f72c6', 'request_id': 'req-625ce616-04b2-444e-8f74-8b84a336c8e5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.887029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b6c8a973f0d4c04b4010302750f72c6', 'request_id': 'req-625ce616-04b2-444e-8f74-8b84a336c8e5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.897649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] The token cache is being invalidate because project 5b6c8a973f0d4c04b4010302750f72c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:31.897932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.898577 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-625ce616-04b2-444e-8f74-8b84a336c8e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:31.901745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 891/1770] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:31 2022] DELETE /identity/v3/projects/5b6c8a973f0d4c04b4010302750f72c6 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:31.905625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad2417-0b70-4214-be05-b80056a9c783 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:31.905895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad2417-0b70-4214-be05-b80056a9c783 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:31.906173 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad2417-0b70-4214-be05-b80056a9c783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:31.929141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-72ad2417-0b70-4214-be05-b80056a9c783 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:31.930023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-72ad2417-0b70-4214-be05-b80056a9c783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:31.946015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 880/1771] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:31.947722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47306dbe-d6c6-4e56-81a5-44fc881efb84 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:31.948026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47306dbe-d6c6-4e56-81a5-44fc881efb84 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:31.948333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47306dbe-d6c6-4e56-81a5-44fc881efb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:31.972530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-47306dbe-d6c6-4e56-81a5-44fc881efb84 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:31.973868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-47306dbe-d6c6-4e56-81a5-44fc881efb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:31.986005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 892/1772] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:31.994417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad2417-0b70-4214-be05-b80056a9c783 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.007734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad2417-0b70-4214-be05-b80056a9c783 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.014527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.015231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.015703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.016043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.016782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:32.017151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.018799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.035022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 881/1773] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:27:32.038443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47306dbe-d6c6-4e56-81a5-44fc881efb84 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.049640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47306dbe-d6c6-4e56-81a5-44fc881efb84 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.055027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.055633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.055914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.056192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.056873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.058764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.159204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 893/1774] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:32 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:32.160337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.170736 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990427e-382a-41a4-8429-23f5169c722b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.213185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.215485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:32.247495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc0992-a9c8-4479-ac22-ccba45bd60e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.248165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0992-a9c8-4479-ac22-ccba45bd60e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.248872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0992-a9c8-4479-ac22-ccba45bd60e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.249156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0992-a9c8-4479-ac22-ccba45bd60e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.249950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc0992-a9c8-4479-ac22-ccba45bd60e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.251755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc0992-a9c8-4479-ac22-ccba45bd60e1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.282237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4504ee5-3ed7-4679-ac21-0f0ab1664eda admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.287135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.287770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.288032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.288331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] PATH_INFO: `/v3/projects/dd0e49015850403ea4e718fb8602b9ff` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.289433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0e49015850403ea4e718fb8602b9ff)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.291457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.325822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 882/1775] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:32 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:32.330053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc0992-a9c8-4479-ac22-ccba45bd60e1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.340307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc0992-a9c8-4479-ac22-ccba45bd60e1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.345025 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba8753-4ad1-41c7-a0ac-cd3f47266267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.345598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba8753-4ad1-41c7-a0ac-cd3f47266267 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.345879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba8753-4ad1-41c7-a0ac-cd3f47266267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.346153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba8753-4ad1-41c7-a0ac-cd3f47266267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.346834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba8753-4ad1-41c7-a0ac-cd3f47266267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.348456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba8753-4ad1-41c7-a0ac-cd3f47266267 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.360309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 883/1776] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:32.362860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba8753-4ad1-41c7-a0ac-cd3f47266267 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.372651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba8753-4ad1-41c7-a0ac-cd3f47266267 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.377286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bd227-3846-449e-8194-eeef6062d787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.377861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bd227-3846-449e-8194-eeef6062d787 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.378207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bd227-3846-449e-8194-eeef6062d787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.378414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bd227-3846-449e-8194-eeef6062d787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.379033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026bd227-3846-449e-8194-eeef6062d787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.380644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026bd227-3846-449e-8194-eeef6062d787 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.390557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0e49015850403ea4e718fb8602b9ff', 'request_id': 'req-f85e062f-85c2-4880-a741-c66ea9c86f0c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.391135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 884/1777] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:32.395549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bd227-3846-449e-8194-eeef6062d787 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.406227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bd227-3846-449e-8194-eeef6062d787 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.452045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:32.461801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56ec8f-8efe-495f-88f0-bb56fee3bbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.462389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ec8f-8efe-495f-88f0-bb56fee3bbb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.462507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0e49015850403ea4e718fb8602b9ff', 'request_id': 'req-f85e062f-85c2-4880-a741-c66ea9c86f0c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.462669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ec8f-8efe-495f-88f0-bb56fee3bbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.462943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56ec8f-8efe-495f-88f0-bb56fee3bbb6 admin admin] PATH_INFO: `/v3/projects/0e847aaaedd940f59efac66104ef8008/users/5c55e66cea624f21a7305de42b9a3948/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.469874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0e49015850403ea4e718fb8602b9ff', 'request_id': 'req-f85e062f-85c2-4880-a741-c66ea9c86f0c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.478247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56ec8f-8efe-495f-88f0-bb56fee3bbb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e847aaaedd940f59efac66104ef8008, user_id=5c55e66cea624f21a7305de42b9a3948, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.480843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] The token cache is being invalidate because project dd0e49015850403ea4e718fb8602b9ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:32.481092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56ec8f-8efe-495f-88f0-bb56fee3bbb6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.481221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.481753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.483441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 894/1778] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:32 2022] DELETE /identity/v3/projects/dd0e49015850403ea4e718fb8602b9ff => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:32.487226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.488550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:32.500872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e062f-85c2-4880-a741-c66ea9c86f0c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.505445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.506030 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.506368 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.506615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] PATH_INFO: `/v3/users/4d7a001022aa426c9c88aca32135e818` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.507687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7a001022aa426c9c88aca32135e818)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.509661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.533528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:32.542343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 885/1779] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:32 2022] PUT /identity/v3/projects/0e847aaaedd940f59efac66104ef8008/users/5c55e66cea624f21a7305de42b9a3948/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:32.547308 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113f209-15d6-4efa-8fa7-c43054328f72 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.547595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113f209-15d6-4efa-8fa7-c43054328f72 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.547871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113f209-15d6-4efa-8fa7-c43054328f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.566621 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a113f209-15d6-4efa-8fa7-c43054328f72 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:32.567802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a113f209-15d6-4efa-8fa7-c43054328f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:32.627104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 886/1780] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:32.629764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7a001022aa426c9c88aca32135e818'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.630869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7a001022aa426c9c88aca32135e818'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.637057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113f209-15d6-4efa-8fa7-c43054328f72 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:32.638520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a113f209-15d6-4efa-8fa7-c43054328f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:32.644235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7a001022aa426c9c88aca32135e818'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.651227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7a001022aa426c9c88aca32135e818'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.651926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7a001022aa426c9c88aca32135e818'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:32.696233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113f209-15d6-4efa-8fa7-c43054328f72 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:32.700476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1fabb-6d0d-42d7-bfe0-1fc7b22abcc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:32.701111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1fabb-6d0d-42d7-bfe0-1fc7b22abcc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:32.701393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1fabb-6d0d-42d7-bfe0-1fc7b22abcc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:32.701667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1fabb-6d0d-42d7-bfe0-1fc7b22abcc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:32.703441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-24b1fabb-6d0d-42d7-bfe0-1fc7b22abcc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:32.716890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1fabb-6d0d-42d7-bfe0-1fc7b22abcc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:32.717021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1fabb-6d0d-42d7-bfe0-1fc7b22abcc6 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:32.718997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 895/1781] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:32 2022] DELETE /identity/v3/users/4d7a001022aa426c9c88aca32135e818 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:32.727170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 887/1782] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:33.156951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:33.160109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:33.253388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e4039-846f-4564-bcec-54096903dbc9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:33.258181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:33.258811 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.259089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.259376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] PATH_INFO: `/v3/projects/4ee9cd6caf244849ba550ecb7a734b55` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.260521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee9cd6caf244849ba550ecb7a734b55)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:33.262459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:33.372123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee9cd6caf244849ba550ecb7a734b55', 'request_id': 'req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:33.439667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee9cd6caf244849ba550ecb7a734b55', 'request_id': 'req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:33.450799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee9cd6caf244849ba550ecb7a734b55', 'request_id': 'req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:33.463086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] The token cache is being invalidate because project 4ee9cd6caf244849ba550ecb7a734b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:33.463383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:33.464053 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ee5c7b20-418a-4c1f-8c14-89b798ce2f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:33.466889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 896/1783] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:33 2022] DELETE /identity/v3/projects/4ee9cd6caf244849ba550ecb7a734b55 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:33.471332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a8acd-2d89-44f6-911d-9e64947439c9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.471830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a8acd-2d89-44f6-911d-9e64947439c9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.472106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a8acd-2d89-44f6-911d-9e64947439c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.491992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a71a8acd-2d89-44f6-911d-9e64947439c9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:33.493267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a71a8acd-2d89-44f6-911d-9e64947439c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:33.551234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 888/1784] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:33.559303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edde78-bd56-4347-8eed-2d537ab29436 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.559518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edde78-bd56-4347-8eed-2d537ab29436 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.559740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edde78-bd56-4347-8eed-2d537ab29436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.580090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-66edde78-bd56-4347-8eed-2d537ab29436 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:33.581332 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-66edde78-bd56-4347-8eed-2d537ab29436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:33.590031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 897/1785] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:33.594228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a8acd-2d89-44f6-911d-9e64947439c9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:33.604160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a8acd-2d89-44f6-911d-9e64947439c9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:33.609429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:33.610028 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.610310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.610584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.611223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:33.611848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:33.613154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:33.630038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 889/1786] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:27:33.631490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edde78-bd56-4347-8eed-2d537ab29436 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:33.640851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edde78-bd56-4347-8eed-2d537ab29436 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:33.645830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461c85a-b825-4b5b-85fa-0a1fb2010d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:33.646489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461c85a-b825-4b5b-85fa-0a1fb2010d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.646876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461c85a-b825-4b5b-85fa-0a1fb2010d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.647207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461c85a-b825-4b5b-85fa-0a1fb2010d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.647832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461c85a-b825-4b5b-85fa-0a1fb2010d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:33.649755 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461c85a-b825-4b5b-85fa-0a1fb2010d90 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:33.742131 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 898/1787] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:33 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:33.743324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:33.749696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:33.753488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f4fe2-dbde-49ee-829b-8d28cf0e1a40 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:33.812551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395233ab-c1cc-452b-9fc7-e3c05c70836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:33.814805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395233ab-c1cc-452b-9fc7-e3c05c70836f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.814805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395233ab-c1cc-452b-9fc7-e3c05c70836f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.814805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395233ab-c1cc-452b-9fc7-e3c05c70836f admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.814805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395233ab-c1cc-452b-9fc7-e3c05c70836f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:33.816574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395233ab-c1cc-452b-9fc7-e3c05c70836f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:33.878623 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 890/1788] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:33 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:33.878726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461c85a-b825-4b5b-85fa-0a1fb2010d90 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:33.884991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:33.891384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461c85a-b825-4b5b-85fa-0a1fb2010d90 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:33.896324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a9d48-e3e3-4f1d-9140-e68ebd698d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:33.896970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a9d48-e3e3-4f1d-9140-e68ebd698d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.901008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a9d48-e3e3-4f1d-9140-e68ebd698d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.901008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a9d48-e3e3-4f1d-9140-e68ebd698d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.901008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a9d48-e3e3-4f1d-9140-e68ebd698d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:33.901008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a9d48-e3e3-4f1d-9140-e68ebd698d2b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:33.910549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 899/1789] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:33.914734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395233ab-c1cc-452b-9fc7-e3c05c70836f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:33.925060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395233ab-c1cc-452b-9fc7-e3c05c70836f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:33.929659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbbf01-9a04-47e0-85a3-c7aba45a478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:33.930393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbf01-9a04-47e0-85a3-c7aba45a478f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.930548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbf01-9a04-47e0-85a3-c7aba45a478f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.930777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbf01-9a04-47e0-85a3-c7aba45a478f admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.931411 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbbf01-9a04-47e0-85a3-c7aba45a478f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:33.933022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbbf01-9a04-47e0-85a3-c7aba45a478f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:33.943190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 891/1790] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:33.947249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a9d48-e3e3-4f1d-9140-e68ebd698d2b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:33.958043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a9d48-e3e3-4f1d-9140-e68ebd698d2b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:33.962797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a598a44-0fce-44f2-85a9-cb42debe916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:33.963439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a598a44-0fce-44f2-85a9-cb42debe916a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:33.963721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a598a44-0fce-44f2-85a9-cb42debe916a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:33.964012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a598a44-0fce-44f2-85a9-cb42debe916a admin admin] PATH_INFO: `/v3/projects/7b67103800a240e588f587ec6df4d35f/users/b2df62347a664e549d07aa460d086c12/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:33.979845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a598a44-0fce-44f2-85a9-cb42debe916a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b67103800a240e588f587ec6df4d35f, user_id=b2df62347a664e549d07aa460d086c12, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:33.982574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a598a44-0fce-44f2-85a9-cb42debe916a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.028970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 900/1791] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:33 2022] PUT /identity/v3/projects/7b67103800a240e588f587ec6df4d35f/users/b2df62347a664e549d07aa460d086c12/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:34.042128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a1bc1-4af9-402c-93d2-8f36ead3d00e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.042519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a1bc1-4af9-402c-93d2-8f36ead3d00e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.042898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a1bc1-4af9-402c-93d2-8f36ead3d00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.081754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76503f29-6ce5-446d-806c-d85ffd5ed796 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.082044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76503f29-6ce5-446d-806c-d85ffd5ed796 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.082324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76503f29-6ce5-446d-806c-d85ffd5ed796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.092179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-162a1bc1-4af9-402c-93d2-8f36ead3d00e None None] MFA Rules not processed for user `b2df62347a664e549d07aa460d086c12`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:34.094987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-162a1bc1-4af9-402c-93d2-8f36ead3d00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:34.113697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-76503f29-6ce5-446d-806c-d85ffd5ed796 None None] MFA Rules not processed for user `5c55e66cea624f21a7305de42b9a3948`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:34.114983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-76503f29-6ce5-446d-806c-d85ffd5ed796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:34.184307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 892/1792] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:34.188432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 901/1793] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:34.195394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a1bc1-4af9-402c-93d2-8f36ead3d00e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.199488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-162a1bc1-4af9-402c-93d2-8f36ead3d00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:34.199606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76503f29-6ce5-446d-806c-d85ffd5ed796 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.199606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-76503f29-6ce5-446d-806c-d85ffd5ed796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:34.267141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76503f29-6ce5-446d-806c-d85ffd5ed796 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.283476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec78cd-57e5-4eaf-ad3a-efdf78f43d1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.283476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec78cd-57e5-4eaf-ad3a-efdf78f43d1c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.283476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec78cd-57e5-4eaf-ad3a-efdf78f43d1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.283476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec78cd-57e5-4eaf-ad3a-efdf78f43d1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.283866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a1bc1-4af9-402c-93d2-8f36ead3d00e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.283967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec78cd-57e5-4eaf-ad3a-efdf78f43d1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.283967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec78cd-57e5-4eaf-ad3a-efdf78f43d1c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.284416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79047866-7c1a-4f70-8693-c1af0311ead8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.285108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79047866-7c1a-4f70-8693-c1af0311ead8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.285475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79047866-7c1a-4f70-8693-c1af0311ead8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.285837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79047866-7c1a-4f70-8693-c1af0311ead8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.286554 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79047866-7c1a-4f70-8693-c1af0311ead8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.288559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79047866-7c1a-4f70-8693-c1af0311ead8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.291536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 902/1794] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:34.314928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec78cd-57e5-4eaf-ad3a-efdf78f43d1c service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.326335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec78cd-57e5-4eaf-ad3a-efdf78f43d1c service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.331548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.332189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.332529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.332796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] PATH_INFO: `/v3/users/b2df62347a664e549d07aa460d086c12` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.333902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2df62347a664e549d07aa460d086c12)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.335847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.366231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:34.377058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 893/1795] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:34 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:34.378019 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79047866-7c1a-4f70-8693-c1af0311ead8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.441476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79047866-7c1a-4f70-8693-c1af0311ead8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.449865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.449865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.449865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.449865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.450244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.450504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.499636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2df62347a664e549d07aa460d086c12'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:34.500329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2df62347a664e549d07aa460d086c12'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:34.505651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:34.513676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2df62347a664e549d07aa460d086c12'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:34.520991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2df62347a664e549d07aa460d086c12'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:34.521667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bd07cbf2-3c84-4d97-bfb1-174fb6eda654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2df62347a664e549d07aa460d086c12'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:34.548820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 894/1796] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:34 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:34.552668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.554376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:34.567314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f41c6-9141-4c0d-a29c-077c875c608e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.572476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fb021-8994-4f93-b53c-a86f72fcc1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.573084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fb021-8994-4f93-b53c-a86f72fcc1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.573360 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fb021-8994-4f93-b53c-a86f72fcc1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.573657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fb021-8994-4f93-b53c-a86f72fcc1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.574301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fb021-8994-4f93-b53c-a86f72fcc1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.575990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fb021-8994-4f93-b53c-a86f72fcc1ca admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.586728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 895/1797] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:34.590912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fb021-8994-4f93-b53c-a86f72fcc1ca admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.600545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fb021-8994-4f93-b53c-a86f72fcc1ca admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.605864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ac9cd-c6db-47d7-8b11-ab30950d3c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.606216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 903/1798] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:34 2022] DELETE /identity/v3/users/b2df62347a664e549d07aa460d086c12 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:34.606505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ac9cd-c6db-47d7-8b11-ab30950d3c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.606779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ac9cd-c6db-47d7-8b11-ab30950d3c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.607052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ac9cd-c6db-47d7-8b11-ab30950d3c14 admin admin] PATH_INFO: `/v3/projects/b1ce17c89f044bb091ed819107f6d5ea/users/973564800262459cba25b09db74c87b3/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.610372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597598-8cbf-405f-81fb-5ff8e2cbec5a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.610660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597598-8cbf-405f-81fb-5ff8e2cbec5a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.610987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597598-8cbf-405f-81fb-5ff8e2cbec5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.623515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ac9cd-c6db-47d7-8b11-ab30950d3c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ce17c89f044bb091ed819107f6d5ea, user_id=973564800262459cba25b09db74c87b3, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.626239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ac9cd-c6db-47d7-8b11-ab30950d3c14 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.631421 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ab597598-8cbf-405f-81fb-5ff8e2cbec5a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:34.632654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ab597598-8cbf-405f-81fb-5ff8e2cbec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:34.690928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 904/1799] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:34.700424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab597598-8cbf-405f-81fb-5ff8e2cbec5a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.701733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ab597598-8cbf-405f-81fb-5ff8e2cbec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:34.702204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 896/1800] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:34 2022] PUT /identity/v3/projects/b1ce17c89f044bb091ed819107f6d5ea/users/973564800262459cba25b09db74c87b3/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:34.706281 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ac9cd-c6db-47d7-8b11-ab30950d3c14 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.760912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab597598-8cbf-405f-81fb-5ff8e2cbec5a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.769559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ac9cd-c6db-47d7-8b11-ab30950d3c14 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.774118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d5201-12dd-4a9d-af28-f51b84bee0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.774721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5201-12dd-4a9d-af28-f51b84bee0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.775001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5201-12dd-4a9d-af28-f51b84bee0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.775274 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5201-12dd-4a9d-af28-f51b84bee0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.775904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d5201-12dd-4a9d-af28-f51b84bee0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.777517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d5201-12dd-4a9d-af28-f51b84bee0b2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.787460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 897/1801] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:34.791733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d5201-12dd-4a9d-af28-f51b84bee0b2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:34.802126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d5201-12dd-4a9d-af28-f51b84bee0b2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:34.806904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.807546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.807947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.808301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] PATH_INFO: `/v3/projects/b1ce17c89f044bb091ed819107f6d5ea/users/973564800262459cba25b09db74c87b3/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.811122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ce17c89f044bb091ed819107f6d5ea, user_id=973564800262459cba25b09db74c87b3, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.811314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12308bf0-a346-4fc6-909f-35db3ea8ba4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:34.811921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12308bf0-a346-4fc6-909f-35db3ea8ba4f service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:34.812203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12308bf0-a346-4fc6-909f-35db3ea8ba4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:34.812508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12308bf0-a346-4fc6-909f-35db3ea8ba4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:34.813821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.822930 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12308bf0-a346-4fc6-909f-35db3ea8ba4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:34.824459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12308bf0-a346-4fc6-909f-35db3ea8ba4f service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:34.825711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 898/1802] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:34 2022] PUT /identity/v3/projects/b1ce17c89f044bb091ed819107f6d5ea/users/973564800262459cba25b09db74c87b3/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:34.835641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 905/1803] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:35.371702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:35.373526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:35.418748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:35.444805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8c961-b5f5-490e-8e9e-9efe10701216 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:35.451315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:35.452076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:35.452467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:35.453236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] PATH_INFO: `/v3/projects/7b67103800a240e588f587ec6df4d35f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:35.457578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b67103800a240e588f587ec6df4d35f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:35.457578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:35.600045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b67103800a240e588f587ec6df4d35f', 'request_id': 'req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:35.605841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b67103800a240e588f587ec6df4d35f', 'request_id': 'req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:35.612451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b67103800a240e588f587ec6df4d35f', 'request_id': 'req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:35.680011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] The token cache is being invalidate because project 7b67103800a240e588f587ec6df4d35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:35.680588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:35.681588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-34f2b575-e784-4f13-ba07-1bdf3f7e6ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:35.686913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 899/1804] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:35 2022] DELETE /identity/v3/projects/7b67103800a240e588f587ec6df4d35f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:35.688890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799185c3-6dd4-434c-8bff-e78fc9480fcc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:35.689369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799185c3-6dd4-434c-8bff-e78fc9480fcc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:35.689471 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799185c3-6dd4-434c-8bff-e78fc9480fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:35.709662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-799185c3-6dd4-434c-8bff-e78fc9480fcc None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:35.710844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-799185c3-6dd4-434c-8bff-e78fc9480fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:35.737283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:35.772261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 906/1805] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:35.780142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e385d8-b83d-4c4e-bc87-69580ca74d35 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:35.780561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e385d8-b83d-4c4e-bc87-69580ca74d35 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:35.780846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e385d8-b83d-4c4e-bc87-69580ca74d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:35.800408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a0e385d8-b83d-4c4e-bc87-69580ca74d35 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:35.801608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0e385d8-b83d-4c4e-bc87-69580ca74d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:35.810192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 900/1806] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:35.814530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799185c3-6dd4-434c-8bff-e78fc9480fcc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:35.824770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799185c3-6dd4-434c-8bff-e78fc9480fcc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:35.829697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:35.830284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:35.830559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:35.830830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:35.831461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:35.831787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:35.833440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:35.845113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 907/1807] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:27:35.849456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e385d8-b83d-4c4e-bc87-69580ca74d35 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:35.859067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e385d8-b83d-4c4e-bc87-69580ca74d35 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:35.864027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cfbbd-fe71-461a-89e1-e96e5049ab03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:35.864671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cfbbd-fe71-461a-89e1-e96e5049ab03 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:35.864948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cfbbd-fe71-461a-89e1-e96e5049ab03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:35.865219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682cfbbd-fe71-461a-89e1-e96e5049ab03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:35.865840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cfbbd-fe71-461a-89e1-e96e5049ab03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:35.867761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682cfbbd-fe71-461a-89e1-e96e5049ab03 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:35.992008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 901/1808] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:35 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:35.994238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.004708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cc3a9-5c51-4f15-88de-d2ca035a9e10 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.061853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a63e05-ddd6-4b6b-afa0-ae364e8802be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.062493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63e05-ddd6-4b6b-afa0-ae364e8802be admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.062769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63e05-ddd6-4b6b-afa0-ae364e8802be admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.063043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a63e05-ddd6-4b6b-afa0-ae364e8802be admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.063668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a63e05-ddd6-4b6b-afa0-ae364e8802be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.065637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a63e05-ddd6-4b6b-afa0-ae364e8802be admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.160258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cfbbd-fe71-461a-89e1-e96e5049ab03 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.160531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 908/1809] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:35 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:36.172905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682cfbbd-fe71-461a-89e1-e96e5049ab03 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.177978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913496f9-357a-4646-b0ad-ce2cacb363f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.179346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4404295-9725-43ae-9d5d-b3ea29269798 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.179595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4404295-9725-43ae-9d5d-b3ea29269798 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.179870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4404295-9725-43ae-9d5d-b3ea29269798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.180844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913496f9-357a-4646-b0ad-ce2cacb363f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.181137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913496f9-357a-4646-b0ad-ce2cacb363f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.181412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913496f9-357a-4646-b0ad-ce2cacb363f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.184729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913496f9-357a-4646-b0ad-ce2cacb363f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.187160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913496f9-357a-4646-b0ad-ce2cacb363f0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.201993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 902/1810] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:36.212121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e4404295-9725-43ae-9d5d-b3ea29269798 None None] MFA Rules not processed for user `973564800262459cba25b09db74c87b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:36.214458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e4404295-9725-43ae-9d5d-b3ea29269798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:36.223186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913496f9-357a-4646-b0ad-ce2cacb363f0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.233927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913496f9-357a-4646-b0ad-ce2cacb363f0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.238953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcbc62-05a2-45b5-bf6e-09217d663ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.239560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcbc62-05a2-45b5-bf6e-09217d663ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.239842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcbc62-05a2-45b5-bf6e-09217d663ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.240140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcbc62-05a2-45b5-bf6e-09217d663ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.240787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcbc62-05a2-45b5-bf6e-09217d663ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.242378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcbc62-05a2-45b5-bf6e-09217d663ec6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.253687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 903/1811] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:36.257613 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcbc62-05a2-45b5-bf6e-09217d663ec6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.269345 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcbc62-05a2-45b5-bf6e-09217d663ec6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.275494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9581ecc-a94a-4913-839a-4a4460e1d875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.277059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9581ecc-a94a-4913-839a-4a4460e1d875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.277059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9581ecc-a94a-4913-839a-4a4460e1d875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.277059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9581ecc-a94a-4913-839a-4a4460e1d875 admin admin] PATH_INFO: `/v3/projects/7410f317be1b45ef98842e39affe5ded/users/8266f0cadad84d17b92ecb8203e8fa1c/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.295736 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9581ecc-a94a-4913-839a-4a4460e1d875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7410f317be1b45ef98842e39affe5ded, user_id=8266f0cadad84d17b92ecb8203e8fa1c, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.302600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9581ecc-a94a-4913-839a-4a4460e1d875 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.309398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 909/1812] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:36.320859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4404295-9725-43ae-9d5d-b3ea29269798 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.322264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e4404295-9725-43ae-9d5d-b3ea29269798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:36.358822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:36.381974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4404295-9725-43ae-9d5d-b3ea29269798 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.385634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.386329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.387549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.387832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.389579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:36.399910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 904/1813] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:36 2022] PUT /identity/v3/projects/7410f317be1b45ef98842e39affe5ded/users/8266f0cadad84d17b92ecb8203e8fa1c/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:36.410865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f54607-cd66-4232-81d6-c2c4fbab2c7f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.411152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f54607-cd66-4232-81d6-c2c4fbab2c7f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.411488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f54607-cd66-4232-81d6-c2c4fbab2c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.439521 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-04f54607-cd66-4232-81d6-c2c4fbab2c7f None None] MFA Rules not processed for user `8266f0cadad84d17b92ecb8203e8fa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:36.440774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-04f54607-cd66-4232-81d6-c2c4fbab2c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:36.453248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.454936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.463663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 910/1814] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:36.491849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:36.516447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 905/1815] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:36.521057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.532187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f85814-c000-4df0-945b-cd3b4834e03b service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.588369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cba671-4696-44b3-a7c2-14e179218e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.588894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cba671-4696-44b3-a7c2-14e179218e47 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.589172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cba671-4696-44b3-a7c2-14e179218e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.589976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cba671-4696-44b3-a7c2-14e179218e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.590670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cba671-4696-44b3-a7c2-14e179218e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.592579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cba671-4696-44b3-a7c2-14e179218e47 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.672680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 911/1816] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:36 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:36.673931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f54607-cd66-4232-81d6-c2c4fbab2c7f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.684600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f54607-cd66-4232-81d6-c2c4fbab2c7f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.744933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3bfda-4f63-47fc-b252-0825ece87ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.745588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3bfda-4f63-47fc-b252-0825ece87ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.745959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3bfda-4f63-47fc-b252-0825ece87ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.746292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3bfda-4f63-47fc-b252-0825ece87ecd admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.746921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3bfda-4f63-47fc-b252-0825ece87ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.748892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3bfda-4f63-47fc-b252-0825ece87ecd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.835728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 906/1817] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:36 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:36.849559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cba671-4696-44b3-a7c2-14e179218e47 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.852815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cba671-4696-44b3-a7c2-14e179218e47 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.857462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f9d8c-592f-4591-af97-ecfc9cef6b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.858061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9d8c-592f-4591-af97-ecfc9cef6b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.858365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9d8c-592f-4591-af97-ecfc9cef6b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.858634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f9d8c-592f-4591-af97-ecfc9cef6b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.859267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f9d8c-592f-4591-af97-ecfc9cef6b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.861099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f9d8c-592f-4591-af97-ecfc9cef6b39 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.878275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 912/1818] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:36.882264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3bfda-4f63-47fc-b252-0825ece87ecd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.892685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3bfda-4f63-47fc-b252-0825ece87ecd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.897219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29001ed8-d631-4a62-a586-8758233fa46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.897804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29001ed8-d631-4a62-a586-8758233fa46b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.898111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29001ed8-d631-4a62-a586-8758233fa46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.898383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29001ed8-d631-4a62-a586-8758233fa46b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.899013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29001ed8-d631-4a62-a586-8758233fa46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.901162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29001ed8-d631-4a62-a586-8758233fa46b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:36.912359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 907/1819] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:36.916859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f9d8c-592f-4591-af97-ecfc9cef6b39 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:36.928567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f9d8c-592f-4591-af97-ecfc9cef6b39 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:36.932553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c8615-397c-4f89-bd82-f31b0e678a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:36.933203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c8615-397c-4f89-bd82-f31b0e678a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:36.933477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c8615-397c-4f89-bd82-f31b0e678a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:36.933743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c8615-397c-4f89-bd82-f31b0e678a86 admin admin] PATH_INFO: `/v3/projects/bbf51e400ebb4d5886ffb251ca48625d/users/a93ff2ea73304b348290abe94cbc4ada/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:36.948237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c8615-397c-4f89-bd82-f31b0e678a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf51e400ebb4d5886ffb251ca48625d, user_id=a93ff2ea73304b348290abe94cbc4ada, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:36.951365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c8615-397c-4f89-bd82-f31b0e678a86 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:37.000005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 913/1820] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:36 2022] PUT /identity/v3/projects/bbf51e400ebb4d5886ffb251ca48625d/users/a93ff2ea73304b348290abe94cbc4ada/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:37.016317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c651992-a850-4445-94a5-42cb93b03836 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.016798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c651992-a850-4445-94a5-42cb93b03836 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.017373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c651992-a850-4445-94a5-42cb93b03836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.046755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3c651992-a850-4445-94a5-42cb93b03836 None None] MFA Rules not processed for user `a93ff2ea73304b348290abe94cbc4ada`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:37.047963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c651992-a850-4445-94a5-42cb93b03836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:37.117599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 908/1821] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:37.122365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c8615-397c-4f89-bd82-f31b0e678a86 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:37.163333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:37.183972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c8615-397c-4f89-bd82-f31b0e678a86 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:37.188750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295b41c-f529-48d8-bd9d-16ee304c02e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:37.189348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b41c-f529-48d8-bd9d-16ee304c02e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.189627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b41c-f529-48d8-bd9d-16ee304c02e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.189897 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b41c-f529-48d8-bd9d-16ee304c02e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.190520 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295b41c-f529-48d8-bd9d-16ee304c02e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:37.192431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295b41c-f529-48d8-bd9d-16ee304c02e3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:37.267284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 914/1822] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:37 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:37.268503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c651992-a850-4445-94a5-42cb93b03836 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:37.279198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c651992-a850-4445-94a5-42cb93b03836 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:37.322741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:37.335032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377161b2-6d68-4b65-9296-6dc95ba4679c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:37.335624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377161b2-6d68-4b65-9296-6dc95ba4679c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.335901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377161b2-6d68-4b65-9296-6dc95ba4679c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.336171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377161b2-6d68-4b65-9296-6dc95ba4679c admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.336843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377161b2-6d68-4b65-9296-6dc95ba4679c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:37.338733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377161b2-6d68-4b65-9296-6dc95ba4679c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:37.399101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 909/1823] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:37 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:37.402948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295b41c-f529-48d8-bd9d-16ee304c02e3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:37.413068 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295b41c-f529-48d8-bd9d-16ee304c02e3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:37.418102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6b914-11a4-4389-aba3-33205a883741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:37.418745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b914-11a4-4389-aba3-33205a883741 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.419027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b914-11a4-4389-aba3-33205a883741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.419305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6b914-11a4-4389-aba3-33205a883741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.419950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6b914-11a4-4389-aba3-33205a883741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:37.421564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6b914-11a4-4389-aba3-33205a883741 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:37.431881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 915/1824] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:37.436465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377161b2-6d68-4b65-9296-6dc95ba4679c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:37.446584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377161b2-6d68-4b65-9296-6dc95ba4679c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:37.451818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e5976-94ac-4da6-aef6-901bad6fddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:37.452582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e5976-94ac-4da6-aef6-901bad6fddfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.452863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e5976-94ac-4da6-aef6-901bad6fddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.453138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e5976-94ac-4da6-aef6-901bad6fddfd admin admin] PATH_INFO: `/v3/projects/034fd501ab084cf8acfd41c8fca50407/users/71e02c187fd04e888ecf352908f8269c/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.468036 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e5976-94ac-4da6-aef6-901bad6fddfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=034fd501ab084cf8acfd41c8fca50407, user_id=71e02c187fd04e888ecf352908f8269c, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:37.470790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e5976-94ac-4da6-aef6-901bad6fddfd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:37.531874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 910/1825] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:37 2022] PUT /identity/v3/projects/034fd501ab084cf8acfd41c8fca50407/users/71e02c187fd04e888ecf352908f8269c/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:37.535853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6b914-11a4-4389-aba3-33205a883741 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:37.546618 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6b914-11a4-4389-aba3-33205a883741 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:37.601562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1d69a-dac1-4611-b55c-2a7457490075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:37.602154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1d69a-dac1-4611-b55c-2a7457490075 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.602432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1d69a-dac1-4611-b55c-2a7457490075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.602704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1d69a-dac1-4611-b55c-2a7457490075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.603330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1d69a-dac1-4611-b55c-2a7457490075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:37.605469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1d69a-dac1-4611-b55c-2a7457490075 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:37.614936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 916/1826] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:37.619244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e5976-94ac-4da6-aef6-901bad6fddfd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:37.629978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e5976-94ac-4da6-aef6-901bad6fddfd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:37.634465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62d482-a9a8-454f-8166-fe1691e0cc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:37.635095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62d482-a9a8-454f-8166-fe1691e0cc4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.635369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62d482-a9a8-454f-8166-fe1691e0cc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.635635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62d482-a9a8-454f-8166-fe1691e0cc4a admin admin] PATH_INFO: `/v3/projects/034fd501ab084cf8acfd41c8fca50407/users/71e02c187fd04e888ecf352908f8269c/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.637537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62d482-a9a8-454f-8166-fe1691e0cc4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=034fd501ab084cf8acfd41c8fca50407, user_id=71e02c187fd04e888ecf352908f8269c, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:37.640123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62d482-a9a8-454f-8166-fe1691e0cc4a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:37.650112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 911/1827] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:37 2022] PUT /identity/v3/projects/034fd501ab084cf8acfd41c8fca50407/users/71e02c187fd04e888ecf352908f8269c/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:37.659909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85203cc0-d63c-4299-b217-c079dbfc2d36 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.660224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85203cc0-d63c-4299-b217-c079dbfc2d36 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.660558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85203cc0-d63c-4299-b217-c079dbfc2d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.690941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-85203cc0-d63c-4299-b217-c079dbfc2d36 None None] MFA Rules not processed for user `71e02c187fd04e888ecf352908f8269c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:37.692135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85203cc0-d63c-4299-b217-c079dbfc2d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:37.759597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 917/1828] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:37.771416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62d482-a9a8-454f-8166-fe1691e0cc4a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:37.826661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62d482-a9a8-454f-8166-fe1691e0cc4a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:37.830526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:37.831118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:37.831392 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:37.831658 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:37.887771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:37.889208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:37.898353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 912/1829] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:41.276541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85203cc0-d63c-4299-b217-c079dbfc2d36 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:41.286609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85203cc0-d63c-4299-b217-c079dbfc2d36 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:41.290114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a488c55-721f-4619-8f85-4c83832eaf65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:41.290700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a488c55-721f-4619-8f85-4c83832eaf65 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:41.290980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a488c55-721f-4619-8f85-4c83832eaf65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:41.291253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a488c55-721f-4619-8f85-4c83832eaf65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:41.320207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:41.350318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a488c55-721f-4619-8f85-4c83832eaf65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:41.351744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a488c55-721f-4619-8f85-4c83832eaf65 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:41.361019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 918/1830] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:47.894099 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:49.370833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:49.372843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:49.383620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77092aed-3313-42ea-ab8c-e29fb5b42e0c service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:49.387478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:49.388616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:49.388896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:49.389170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:49.446347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:49.447759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:49.457321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 913/1831] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:51.101183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a488c55-721f-4619-8f85-4c83832eaf65 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:51.110148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a488c55-721f-4619-8f85-4c83832eaf65 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:51.113827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aab68-c595-4c69-b935-4de0afbde35f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:51.114355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aab68-c595-4c69-b935-4de0afbde35f service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:51.114658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aab68-c595-4c69-b935-4de0afbde35f service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:51.114929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aab68-c595-4c69-b935-4de0afbde35f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:51.124188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175aab68-c595-4c69-b935-4de0afbde35f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:51.126263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175aab68-c595-4c69-b935-4de0afbde35f service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:51.135892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 919/1832] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:53.338686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:53.338686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:53.414557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca78a2-ffa7-484e-b78d-09a7f36df276 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:53.419522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:53.420476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:53.420817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:53.422154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] PATH_INFO: `/v3/users/6cd0df1355a846dab1eb97712f8d3abd` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:53.423333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cd0df1355a846dab1eb97712f8d3abd)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:53.425440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:53.601020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cd0df1355a846dab1eb97712f8d3abd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:53.615143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cd0df1355a846dab1eb97712f8d3abd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:53.628659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cd0df1355a846dab1eb97712f8d3abd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:53.724652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cd0df1355a846dab1eb97712f8d3abd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:53.725302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cd0df1355a846dab1eb97712f8d3abd'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:53.732015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 914/1833] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:53 2022] DELETE /identity/v3/users/6cd0df1355a846dab1eb97712f8d3abd => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:53.736997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b030043-c6bf-4209-accb-72fa5a9fe9fa None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:53.737512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b030043-c6bf-4209-accb-72fa5a9fe9fa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:53.737606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b030043-c6bf-4209-accb-72fa5a9fe9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:53.756808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3b030043-c6bf-4209-accb-72fa5a9fe9fa None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:53.762680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3b030043-c6bf-4209-accb-72fa5a9fe9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:53.822234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 920/1834] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:53.832141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:53.834426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:53.856142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:53.892739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41b3db-1620-425b-9e61-90a3f0db1150 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:53.896310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:53.896918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:53.897213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:53.897464 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:53.906898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:53.908341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:53.917597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 915/1835] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:53.988310 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b030043-c6bf-4209-accb-72fa5a9fe9fa None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:53.989750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3b030043-c6bf-4209-accb-72fa5a9fe9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:54.014143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b030043-c6bf-4209-accb-72fa5a9fe9fa None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:54.041103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:54.041751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:54.042029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:54.042301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] PATH_INFO: `/v3/users/cf1420e3bb664fc0b0df46984fde385c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:54.043561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1420e3bb664fc0b0df46984fde385c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:54.045735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:54.291059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1420e3bb664fc0b0df46984fde385c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.291793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1420e3bb664fc0b0df46984fde385c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.307788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1420e3bb664fc0b0df46984fde385c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.313780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1420e3bb664fc0b0df46984fde385c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.314426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c772e897-c5e2-4b70-9be3-71f726dbc943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1420e3bb664fc0b0df46984fde385c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.365639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:54.367110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:54.382256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b4e74-6544-4acf-8479-6ad46beb1f37 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:54.382693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 921/1836] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:53 2022] DELETE /identity/v3/users/cf1420e3bb664fc0b0df46984fde385c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:54.390750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e49639-13a3-47de-b0db-6db671a9349a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:54.390750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e49639-13a3-47de-b0db-6db671a9349a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:54.390750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e49639-13a3-47de-b0db-6db671a9349a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:54.426327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d8e49639-13a3-47de-b0db-6db671a9349a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:54.427892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d8e49639-13a3-47de-b0db-6db671a9349a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:54.471462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:54.514990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 922/1837] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:54.527948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e49639-13a3-47de-b0db-6db671a9349a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:54.533989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:54.534218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d8e49639-13a3-47de-b0db-6db671a9349a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:54.534300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:54.534300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:54.534300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] PATH_INFO: `/v3/projects/ec02fa1dbaa344c79caefbe2727f26af` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:54.534300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec02fa1dbaa344c79caefbe2727f26af)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:54.536457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:54.604837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e49639-13a3-47de-b0db-6db671a9349a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:54.608405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cc6d6-2080-4886-affc-133713316c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:54.609028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc6d6-2080-4886-affc-133713316c1d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:54.609304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc6d6-2080-4886-affc-133713316c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:54.609583 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc6d6-2080-4886-affc-133713316c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:54.618238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cc6d6-2080-4886-affc-133713316c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:54.619660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cc6d6-2080-4886-affc-133713316c1d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:54.629263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 923/1838] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:54.702655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec02fa1dbaa344c79caefbe2727f26af', 'request_id': 'req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.709167 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec02fa1dbaa344c79caefbe2727f26af', 'request_id': 'req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.715931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec02fa1dbaa344c79caefbe2727f26af', 'request_id': 'req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.782921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] The token cache is being invalidate because project ec02fa1dbaa344c79caefbe2727f26af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:54.783208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.783831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-21f0b999-7bc7-47cb-b1fe-a6da747d76fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:54.785379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 916/1839] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:54 2022] DELETE /identity/v3/projects/ec02fa1dbaa344c79caefbe2727f26af => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:54.790354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d6e2f-2503-4c24-972e-f9c168c7b5db None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:54.790838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d6e2f-2503-4c24-972e-f9c168c7b5db None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:54.790960 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d6e2f-2503-4c24-972e-f9c168c7b5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:54.810954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fe6d6e2f-2503-4c24-972e-f9c168c7b5db None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:54.812201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe6d6e2f-2503-4c24-972e-f9c168c7b5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:54.880789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 924/1840] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:54.889299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c87c4-5af7-4e74-a2f0-79b7cf592cb5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:54.889904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c87c4-5af7-4e74-a2f0-79b7cf592cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:54.890684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c87c4-5af7-4e74-a2f0-79b7cf592cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:54.911472 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b85c87c4-5af7-4e74-a2f0-79b7cf592cb5 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:54.913928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b85c87c4-5af7-4e74-a2f0-79b7cf592cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:54.922189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 917/1841] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:54.926754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d6e2f-2503-4c24-972e-f9c168c7b5db None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:54.936359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d6e2f-2503-4c24-972e-f9c168c7b5db None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:54.940918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:54.941470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:54.945847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:54.945847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:54.945847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:54.945847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:54.945847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:54.958171 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 925/1842] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:27:54.962452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c87c4-5af7-4e74-a2f0-79b7cf592cb5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:54.972321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c87c4-5af7-4e74-a2f0-79b7cf592cb5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:54.976956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3db00-f4b2-49d9-994f-3362ab37e211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:54.977549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3db00-f4b2-49d9-994f-3362ab37e211 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:54.977826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3db00-f4b2-49d9-994f-3362ab37e211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:54.978096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3db00-f4b2-49d9-994f-3362ab37e211 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:54.978717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3db00-f4b2-49d9-994f-3362ab37e211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:54.980776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3db00-f4b2-49d9-994f-3362ab37e211 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.018416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.020105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:55.034432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729e804-e100-463c-817c-f18f2522922f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.039275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.039905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.040180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.041049 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] PATH_INFO: `/v3/projects/058eabdc497c4ab9abbb31f74656ae71` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.042294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=058eabdc497c4ab9abbb31f74656ae71)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.044141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.081068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 918/1843] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:54 2022] POST /identity/v3/projects => generated 361 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:55.085360 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3db00-f4b2-49d9-994f-3362ab37e211 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.110971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:55.136221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058eabdc497c4ab9abbb31f74656ae71', 'request_id': 'req-03dbf510-e077-41fd-bd2c-78df576b5cfe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:55.138735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:55.150638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3db00-f4b2-49d9-994f-3362ab37e211 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.205898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.206425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.206700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.206968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.207611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.209530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.216054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058eabdc497c4ab9abbb31f74656ae71', 'request_id': 'req-03dbf510-e077-41fd-bd2c-78df576b5cfe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:55.223782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058eabdc497c4ab9abbb31f74656ae71', 'request_id': 'req-03dbf510-e077-41fd-bd2c-78df576b5cfe'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:55.233966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] The token cache is being invalidate because project 058eabdc497c4ab9abbb31f74656ae71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:55.234249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:55.235407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-03dbf510-e077-41fd-bd2c-78df576b5cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:55.236946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 926/1844] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:55 2022] DELETE /identity/v3/projects/058eabdc497c4ab9abbb31f74656ae71 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:55.241881 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154d8ff-ee94-4a06-b77b-5afbb1284bde None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.242161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154d8ff-ee94-4a06-b77b-5afbb1284bde None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.242427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154d8ff-ee94-4a06-b77b-5afbb1284bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.261732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-b154d8ff-ee94-4a06-b77b-5afbb1284bde None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:55.262942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-b154d8ff-ee94-4a06-b77b-5afbb1284bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:55.271532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 927/1845] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:55.279052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0a93b-d9c9-4a32-b697-1b9194e4a505 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.279331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0a93b-d9c9-4a32-b697-1b9194e4a505 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.279598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0a93b-d9c9-4a32-b697-1b9194e4a505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.298338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-11c0a93b-d9c9-4a32-b697-1b9194e4a505 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:55.299498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-11c0a93b-d9c9-4a32-b697-1b9194e4a505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:55.307574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 928/1846] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:55.311585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0a93b-d9c9-4a32-b697-1b9194e4a505 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.313833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 919/1847] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:55 2022] POST /identity/v3/users => generated 281 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:55.317878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.319517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:55.321020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0a93b-d9c9-4a32-b697-1b9194e4a505 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.325568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.326159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.326433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.326708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.327338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:55.329035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.330134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.338732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55ef7e-6de8-415a-a6ff-ca34222160a8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.338732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e00059-6ff3-4709-a332-fde10ebfa32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.338732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00059-6ff3-4709-a332-fde10ebfa32e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.338732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00059-6ff3-4709-a332-fde10ebfa32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.339101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e00059-6ff3-4709-a332-fde10ebfa32e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.339101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e00059-6ff3-4709-a332-fde10ebfa32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.340388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e00059-6ff3-4709-a332-fde10ebfa32e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.341784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 929/1848] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:27:55.346169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.353457 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 920/1849] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:55.354402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e00059-6ff3-4709-a332-fde10ebfa32e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.355612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441a28d-ecaf-4bf7-a592-c3714c01e2d0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.360606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980751c8-e1d1-4886-a152-e25058617379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.361197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980751c8-e1d1-4886-a152-e25058617379 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.361528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980751c8-e1d1-4886-a152-e25058617379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.362007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980751c8-e1d1-4886-a152-e25058617379 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.362620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980751c8-e1d1-4886-a152-e25058617379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.364176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e00059-6ff3-4709-a332-fde10ebfa32e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.364556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980751c8-e1d1-4886-a152-e25058617379 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.368962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec4a0-917e-4c6c-8d39-b397665a4984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.369551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec4a0-917e-4c6c-8d39-b397665a4984 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.369831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec4a0-917e-4c6c-8d39-b397665a4984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.370128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec4a0-917e-4c6c-8d39-b397665a4984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.370778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bec4a0-917e-4c6c-8d39-b397665a4984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.372425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bec4a0-917e-4c6c-8d39-b397665a4984 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.383470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 921/1850] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:55.387481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec4a0-917e-4c6c-8d39-b397665a4984 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.397373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec4a0-917e-4c6c-8d39-b397665a4984 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.401930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d76a2-c5fe-4571-89f3-adfb1a9635a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.402535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d76a2-c5fe-4571-89f3-adfb1a9635a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.402927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d76a2-c5fe-4571-89f3-adfb1a9635a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.403131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d76a2-c5fe-4571-89f3-adfb1a9635a3 admin admin] PATH_INFO: `/v3/projects/5f5ff8ce4e5343e38f2dc7a34754f85e/users/eb05de19f4cb476c8c5bc39c662fa1a0/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.417121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d76a2-c5fe-4571-89f3-adfb1a9635a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f5ff8ce4e5343e38f2dc7a34754f85e, user_id=eb05de19f4cb476c8c5bc39c662fa1a0, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.419732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d76a2-c5fe-4571-89f3-adfb1a9635a3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.457415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 930/1851] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:55 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:55.461388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980751c8-e1d1-4886-a152-e25058617379 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.522629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980751c8-e1d1-4886-a152-e25058617379 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.525322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 922/1852] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:55 2022] PUT /identity/v3/projects/5f5ff8ce4e5343e38f2dc7a34754f85e/users/eb05de19f4cb476c8c5bc39c662fa1a0/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:55.535658 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63946537-2f8a-404f-8a6c-fbd0fe7f043a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.535948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63946537-2f8a-404f-8a6c-fbd0fe7f043a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.536469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63946537-2f8a-404f-8a6c-fbd0fe7f043a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.583065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-63946537-2f8a-404f-8a6c-fbd0fe7f043a None None] MFA Rules not processed for user `eb05de19f4cb476c8c5bc39c662fa1a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:55.584500 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-63946537-2f8a-404f-8a6c-fbd0fe7f043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:55.617207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f4b35-8988-4fe2-876b-a8d05ad6dfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.617855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f4b35-8988-4fe2-876b-a8d05ad6dfbb admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.620746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f4b35-8988-4fe2-876b-a8d05ad6dfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.621182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f4b35-8988-4fe2-876b-a8d05ad6dfbb admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.622153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f4b35-8988-4fe2-876b-a8d05ad6dfbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.624251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f4b35-8988-4fe2-876b-a8d05ad6dfbb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.659193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 923/1853] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:55.700583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63946537-2f8a-404f-8a6c-fbd0fe7f043a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.702175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-63946537-2f8a-404f-8a6c-fbd0fe7f043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:55.708006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:55.716931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 931/1854] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:55 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:55.717966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f4b35-8988-4fe2-876b-a8d05ad6dfbb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.730239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f4b35-8988-4fe2-876b-a8d05ad6dfbb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.735558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab2aef-ec13-46d7-8848-ee7c21365ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.736207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab2aef-ec13-46d7-8848-ee7c21365ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.736527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab2aef-ec13-46d7-8848-ee7c21365ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.736798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab2aef-ec13-46d7-8848-ee7c21365ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.737430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab2aef-ec13-46d7-8848-ee7c21365ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.739017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab2aef-ec13-46d7-8848-ee7c21365ef0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.749225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 932/1855] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:55.753742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab2aef-ec13-46d7-8848-ee7c21365ef0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.763693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab2aef-ec13-46d7-8848-ee7c21365ef0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.768293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df938-11e7-4e95-9110-c574b86c85a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.769026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df938-11e7-4e95-9110-c574b86c85a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.769314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df938-11e7-4e95-9110-c574b86c85a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.769580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df938-11e7-4e95-9110-c574b86c85a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.770203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271df938-11e7-4e95-9110-c574b86c85a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.770666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63946537-2f8a-404f-8a6c-fbd0fe7f043a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.771765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271df938-11e7-4e95-9110-c574b86c85a9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.774279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13c621-2e03-4d09-92a8-63566e65cc80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.774880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c621-2e03-4d09-92a8-63566e65cc80 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.775150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c621-2e03-4d09-92a8-63566e65cc80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.775422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c621-2e03-4d09-92a8-63566e65cc80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.782682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 933/1856] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:27:55.784034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13c621-2e03-4d09-92a8-63566e65cc80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.785503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13c621-2e03-4d09-92a8-63566e65cc80 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.786692 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df938-11e7-4e95-9110-c574b86c85a9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.794648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 924/1857] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:55.796225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df938-11e7-4e95-9110-c574b86c85a9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.801481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252d8b4-e039-4cf7-b416-9d394e4595ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.802131 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d8b4-e039-4cf7-b416-9d394e4595ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.802520 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d8b4-e039-4cf7-b416-9d394e4595ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.802674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252d8b4-e039-4cf7-b416-9d394e4595ae admin admin] PATH_INFO: `/v3/projects/1742dccd5a764a31ab1a228ead5cdab9/users/834b92eb61864d1b95e9f037379c2e13/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.805074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:55.811270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13c621-2e03-4d09-92a8-63566e65cc80 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:55.818997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252d8b4-e039-4cf7-b416-9d394e4595ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=1742dccd5a764a31ab1a228ead5cdab9, user_id=834b92eb61864d1b95e9f037379c2e13, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.821690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252d8b4-e039-4cf7-b416-9d394e4595ae admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.822580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13c621-2e03-4d09-92a8-63566e65cc80 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:55.827290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:55.828098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.828994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.829464 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] PATH_INFO: `/v3/users/eb05de19f4cb476c8c5bc39c662fa1a0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.831071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb05de19f4cb476c8c5bc39c662fa1a0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:55.833387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:55.908011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 934/1858] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:55 2022] PUT /identity/v3/projects/1742dccd5a764a31ab1a228ead5cdab9/users/834b92eb61864d1b95e9f037379c2e13/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:55.912887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ffbf0-cad5-4a85-83fe-22987ce9c869 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:55.913168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ffbf0-cad5-4a85-83fe-22987ce9c869 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:55.913436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ffbf0-cad5-4a85-83fe-22987ce9c869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:55.932580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c80ffbf0-cad5-4a85-83fe-22987ce9c869 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:55.933747 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c80ffbf0-cad5-4a85-83fe-22987ce9c869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:55.993635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 935/1859] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:56.003364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ffbf0-cad5-4a85-83fe-22987ce9c869 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:56.004748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c80ffbf0-cad5-4a85-83fe-22987ce9c869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:56.023591 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb05de19f4cb476c8c5bc39c662fa1a0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:56.031619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb05de19f4cb476c8c5bc39c662fa1a0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:56.044150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb05de19f4cb476c8c5bc39c662fa1a0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:56.063565 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ffbf0-cad5-4a85-83fe-22987ce9c869 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:56.067294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d90f2a-4446-4606-a94f-ab5ca1fa77e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:56.067951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d90f2a-4446-4606-a94f-ab5ca1fa77e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:56.068228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d90f2a-4446-4606-a94f-ab5ca1fa77e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:56.068523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d90f2a-4446-4606-a94f-ab5ca1fa77e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:56.079046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d90f2a-4446-4606-a94f-ab5ca1fa77e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:56.080513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d90f2a-4446-4606-a94f-ab5ca1fa77e3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:56.090545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 936/1860] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:27:56.101848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb05de19f4cb476c8c5bc39c662fa1a0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:56.102578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb05de19f4cb476c8c5bc39c662fa1a0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:56.109790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 925/1861] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:55 2022] DELETE /identity/v3/users/eb05de19f4cb476c8c5bc39c662fa1a0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:27:56.114311 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f92219-a0eb-4ba3-ab73-a1544a60bb5c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:56.114593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f92219-a0eb-4ba3-ab73-a1544a60bb5c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:56.114858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f92219-a0eb-4ba3-ab73-a1544a60bb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:56.134012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-06f92219-a0eb-4ba3-ab73-a1544a60bb5c None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:56.135187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-06f92219-a0eb-4ba3-ab73-a1544a60bb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:56.197669 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 937/1862] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:56.209672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:56.213558 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:56.277324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be3eb-11c0-4b90-87f4-025cca3544a6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:56.281121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be570c-0ff8-4da6-af19-dea198833d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:56.281728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be570c-0ff8-4da6-af19-dea198833d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:56.282007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be570c-0ff8-4da6-af19-dea198833d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:56.282278 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be570c-0ff8-4da6-af19-dea198833d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:56.292026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be570c-0ff8-4da6-af19-dea198833d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:56.293481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be570c-0ff8-4da6-af19-dea198833d25 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:56.303933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 926/1863] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:56.871424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f92219-a0eb-4ba3-ab73-a1544a60bb5c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:56.873035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-06f92219-a0eb-4ba3-ab73-a1544a60bb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:56.887204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f92219-a0eb-4ba3-ab73-a1544a60bb5c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:56.894878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:56.895733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:56.896542 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:56.896994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] PATH_INFO: `/v3/projects/5f5ff8ce4e5343e38f2dc7a34754f85e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:56.902889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f5ff8ce4e5343e38f2dc7a34754f85e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:56.905195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:56.935944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:57.033233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f5ff8ce4e5343e38f2dc7a34754f85e', 'request_id': 'req-358f0c4c-443a-4e30-bca1-babcb737149d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:57.113486 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f5ff8ce4e5343e38f2dc7a34754f85e', 'request_id': 'req-358f0c4c-443a-4e30-bca1-babcb737149d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:57.120432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f5ff8ce4e5343e38f2dc7a34754f85e', 'request_id': 'req-358f0c4c-443a-4e30-bca1-babcb737149d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:57.131483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] The token cache is being invalidate because project 5f5ff8ce4e5343e38f2dc7a34754f85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:57.131773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:57.132615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-358f0c4c-443a-4e30-bca1-babcb737149d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:57.133995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 938/1864] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:56 2022] DELETE /identity/v3/projects/5f5ff8ce4e5343e38f2dc7a34754f85e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:57.138952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35c43d-dbef-4a47-9fd3-716fd750d2af None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.139495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35c43d-dbef-4a47-9fd3-716fd750d2af None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.139589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35c43d-dbef-4a47-9fd3-716fd750d2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.161097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9a35c43d-dbef-4a47-9fd3-716fd750d2af None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:57.162728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9a35c43d-dbef-4a47-9fd3-716fd750d2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:57.176647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:57.222598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 927/1865] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:57.231513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e091c-f50e-433e-b429-eccef80e82fc None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.231838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e091c-f50e-433e-b429-eccef80e82fc None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.232159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e091c-f50e-433e-b429-eccef80e82fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.253317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3a6e091c-f50e-433e-b429-eccef80e82fc None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:57.254739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3a6e091c-f50e-433e-b429-eccef80e82fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:57.262519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 939/1866] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:57.267855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35c43d-dbef-4a47-9fd3-716fd750d2af None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:57.277595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35c43d-dbef-4a47-9fd3-716fd750d2af None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:57.282118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:57.282752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.283033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.283304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.283952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:27:57.284233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:57.285846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:57.298653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 928/1867] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:27:57.302995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e091c-f50e-433e-b429-eccef80e82fc None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:57.312919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e091c-f50e-433e-b429-eccef80e82fc None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:57.317470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3e10b-097c-46a3-b856-f78c7b22b30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:57.318050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3e10b-097c-46a3-b856-f78c7b22b30a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.318326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3e10b-097c-46a3-b856-f78c7b22b30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.318592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3e10b-097c-46a3-b856-f78c7b22b30a admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.319208 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3e10b-097c-46a3-b856-f78c7b22b30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:57.321123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3e10b-097c-46a3-b856-f78c7b22b30a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:57.410020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 940/1868] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:57 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:57.411197 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:57.421419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043f38d-85d5-40ce-99e0-8f373ddd9431 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:57.485621 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e29f0-4d84-45bb-a4f1-151656d089bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:57.486223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e29f0-4d84-45bb-a4f1-151656d089bf admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.486502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e29f0-4d84-45bb-a4f1-151656d089bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.486763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e29f0-4d84-45bb-a4f1-151656d089bf admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.487391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e29f0-4d84-45bb-a4f1-151656d089bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:57.489322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e29f0-4d84-45bb-a4f1-151656d089bf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:57.630728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 929/1869] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:57 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:57.632088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3e10b-097c-46a3-b856-f78c7b22b30a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:57.642952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3e10b-097c-46a3-b856-f78c7b22b30a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:57.648757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6f684-781a-4649-86d9-70a143049b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:57.649377 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f684-781a-4649-86d9-70a143049b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.649653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f684-781a-4649-86d9-70a143049b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.649921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f684-781a-4649-86d9-70a143049b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.650554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6f684-781a-4649-86d9-70a143049b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:57.652157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6f684-781a-4649-86d9-70a143049b70 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:57.663461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 941/1870] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:57.667567 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e29f0-4d84-45bb-a4f1-151656d089bf admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:57.679214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e29f0-4d84-45bb-a4f1-151656d089bf admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:57.690361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cde44-5427-4f9f-acf0-8e839eae9068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:57.690988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cde44-5427-4f9f-acf0-8e839eae9068 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.694179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cde44-5427-4f9f-acf0-8e839eae9068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.694179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cde44-5427-4f9f-acf0-8e839eae9068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.694179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cde44-5427-4f9f-acf0-8e839eae9068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:57.694179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cde44-5427-4f9f-acf0-8e839eae9068 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:57.706614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 930/1871] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:57.710961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6f684-781a-4649-86d9-70a143049b70 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:57.722008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6f684-781a-4649-86d9-70a143049b70 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:57.726838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e90c-92a6-4928-95d3-83976a6966e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:57.727494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e90c-92a6-4928-95d3-83976a6966e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.727778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e90c-92a6-4928-95d3-83976a6966e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.728059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703e90c-92a6-4928-95d3-83976a6966e9 admin admin] PATH_INFO: `/v3/projects/a4a17e6e1e7847fbace6d036669df116/users/fc35115b2d3d451aa13267a9d24b2e83/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.744438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703e90c-92a6-4928-95d3-83976a6966e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a17e6e1e7847fbace6d036669df116, user_id=fc35115b2d3d451aa13267a9d24b2e83, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:57.747303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703e90c-92a6-4928-95d3-83976a6966e9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:57.845584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 942/1872] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:57 2022] PUT /identity/v3/projects/a4a17e6e1e7847fbace6d036669df116/users/fc35115b2d3d451aa13267a9d24b2e83/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:57.856308 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312d3e1-de67-4935-9512-7a42a105e887 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:57.856528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312d3e1-de67-4935-9512-7a42a105e887 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:57.856804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312d3e1-de67-4935-9512-7a42a105e887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:57.885870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1312d3e1-de67-4935-9512-7a42a105e887 None None] MFA Rules not processed for user `fc35115b2d3d451aa13267a9d24b2e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:57.887081 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1312d3e1-de67-4935-9512-7a42a105e887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:57.973876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 931/1873] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:27:57.975481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e90c-92a6-4928-95d3-83976a6966e9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:58.055053 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703e90c-92a6-4928-95d3-83976a6966e9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:58.061010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afbae9-d2d6-49f7-9e0a-c87ae2593277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:58.061645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afbae9-d2d6-49f7-9e0a-c87ae2593277 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:58.062648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afbae9-d2d6-49f7-9e0a-c87ae2593277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:58.063051 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afbae9-d2d6-49f7-9e0a-c87ae2593277 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:58.063727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afbae9-d2d6-49f7-9e0a-c87ae2593277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:58.065778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afbae9-d2d6-49f7-9e0a-c87ae2593277 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:58.241336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 943/1874] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:27:57 2022] POST /identity/v3/projects => generated 375 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:27:58.245270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312d3e1-de67-4935-9512-7a42a105e887 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:58.263015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312d3e1-de67-4935-9512-7a42a105e887 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:58.305005 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:58.362870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28869e69-31eb-4af8-a834-af049f0c6245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:58.363256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28869e69-31eb-4af8-a834-af049f0c6245 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:58.363532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28869e69-31eb-4af8-a834-af049f0c6245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:58.363800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28869e69-31eb-4af8-a834-af049f0c6245 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:58.364483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28869e69-31eb-4af8-a834-af049f0c6245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:58.366456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28869e69-31eb-4af8-a834-af049f0c6245 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:58.466436 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 932/1875] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:27:58 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:27:58.466887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afbae9-d2d6-49f7-9e0a-c87ae2593277 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:58.472525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afbae9-d2d6-49f7-9e0a-c87ae2593277 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:58.477293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741da3e-cb17-414e-abc2-8a7609e630ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:58.477900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741da3e-cb17-414e-abc2-8a7609e630ed admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:58.478203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741da3e-cb17-414e-abc2-8a7609e630ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:58.478473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741da3e-cb17-414e-abc2-8a7609e630ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:58.479148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741da3e-cb17-414e-abc2-8a7609e630ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:58.481021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741da3e-cb17-414e-abc2-8a7609e630ed admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:58.498544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 944/1876] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:58.503049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28869e69-31eb-4af8-a834-af049f0c6245 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:58.516623 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28869e69-31eb-4af8-a834-af049f0c6245 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:58.519484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fccf2-d291-4585-b9b1-85749ee5ae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:58.520310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fccf2-d291-4585-b9b1-85749ee5ae57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:58.520657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fccf2-d291-4585-b9b1-85749ee5ae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:58.521000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fccf2-d291-4585-b9b1-85749ee5ae57 admin admin] PATH_INFO: `/v3/projects/a3883eba1ead4c0ca66b6119b9e2ae31/users/54e7743845f24065b49ffdd6373603d5/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:58.544886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fccf2-d291-4585-b9b1-85749ee5ae57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3883eba1ead4c0ca66b6119b9e2ae31, user_id=54e7743845f24065b49ffdd6373603d5, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:58.547867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fccf2-d291-4585-b9b1-85749ee5ae57 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:58.782260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 933/1877] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:58 2022] PUT /identity/v3/projects/a3883eba1ead4c0ca66b6119b9e2ae31/users/54e7743845f24065b49ffdd6373603d5/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:58.790958 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741da3e-cb17-414e-abc2-8a7609e630ed admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:58.800952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741da3e-cb17-414e-abc2-8a7609e630ed admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:58.823099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:58.868758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f7227-dbe7-4c2f-b268-67261919be7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:58.869376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f7227-dbe7-4c2f-b268-67261919be7c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:58.869656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f7227-dbe7-4c2f-b268-67261919be7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:58.869931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f7227-dbe7-4c2f-b268-67261919be7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:58.870567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f7227-dbe7-4c2f-b268-67261919be7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:58.872193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f7227-dbe7-4c2f-b268-67261919be7c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:58.897559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 945/1878] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:27:58.898778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fccf2-d291-4585-b9b1-85749ee5ae57 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:58.914691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fccf2-d291-4585-b9b1-85749ee5ae57 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:58.923420 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:58.926103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:58.926103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:58.926103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] PATH_INFO: `/v3/projects/a3883eba1ead4c0ca66b6119b9e2ae31/users/54e7743845f24065b49ffdd6373603d5/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:58.927066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3883eba1ead4c0ca66b6119b9e2ae31, user_id=54e7743845f24065b49ffdd6373603d5, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:58.933704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:58.952326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 934/1879] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:27:58 2022] PUT /identity/v3/projects/a3883eba1ead4c0ca66b6119b9e2ae31/users/54e7743845f24065b49ffdd6373603d5/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:58.966679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16515446-7251-4630-a1e4-89c7965f6793 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:58.966957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16515446-7251-4630-a1e4-89c7965f6793 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:58.967225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16515446-7251-4630-a1e4-89c7965f6793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:59.002013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-16515446-7251-4630-a1e4-89c7965f6793 None None] MFA Rules not processed for user `54e7743845f24065b49ffdd6373603d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:27:59.003338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-16515446-7251-4630-a1e4-89c7965f6793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:59.091924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 946/1880] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:27:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:27:59.100787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:59.102534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:59.172181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25baf4-0202-420f-aa6e-ba083ad158ce admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:59.175760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:59.176473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:59.176748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:59.177014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:59.242482 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:59.243917 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:59.254572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 935/1881] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:02.375267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115777b9-a4d0-417f-89ef-f39756d46f24 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:02.375729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115777b9-a4d0-417f-89ef-f39756d46f24 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:02.375808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115777b9-a4d0-417f-89ef-f39756d46f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:02.409578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-115777b9-a4d0-417f-89ef-f39756d46f24 None None] MFA Rules not processed for user `834b92eb61864d1b95e9f037379c2e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:02.414025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-115777b9-a4d0-417f-89ef-f39756d46f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:02.489833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 947/1882] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:02.494805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:02.497040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:02.528896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:02.572828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1c802-324a-47ac-b1bf-11847cc5feb7 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:02.577780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0982d-3f6b-48c8-8152-fda554695f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:02.578850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0982d-3f6b-48c8-8152-fda554695f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:02.579127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0982d-3f6b-48c8-8152-fda554695f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:02.579469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0982d-3f6b-48c8-8152-fda554695f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:02.580143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0982d-3f6b-48c8-8152-fda554695f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:02.582150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0982d-3f6b-48c8-8152-fda554695f99 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:02.692590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 936/1883] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:28:02 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:28:02.700898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115777b9-a4d0-417f-89ef-f39756d46f24 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:02.714977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115777b9-a4d0-417f-89ef-f39756d46f24 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:02.751031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:02.779449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88c5a9-28e3-471e-987f-8aa727e97838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:02.780077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88c5a9-28e3-471e-987f-8aa727e97838 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:02.780360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88c5a9-28e3-471e-987f-8aa727e97838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:02.780637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88c5a9-28e3-471e-987f-8aa727e97838 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:02.781272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88c5a9-28e3-471e-987f-8aa727e97838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:02.783200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88c5a9-28e3-471e-987f-8aa727e97838 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:02.891105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 948/1884] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:28:02 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:02.894783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0982d-3f6b-48c8-8152-fda554695f99 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:02.905196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0982d-3f6b-48c8-8152-fda554695f99 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:02.909985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:02.910508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:02.910761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:02.911084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:02.911781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:02.913397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:02.934607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 937/1885] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:28:02.936056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88c5a9-28e3-471e-987f-8aa727e97838 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:02.962564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88c5a9-28e3-471e-987f-8aa727e97838 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:02.964140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6545d-8f53-474a-a241-98bca01cb419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:02.969153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6545d-8f53-474a-a241-98bca01cb419 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:02.969153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6545d-8f53-474a-a241-98bca01cb419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:02.969153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6545d-8f53-474a-a241-98bca01cb419 admin admin] PATH_INFO: `/v3/projects/bb17d85bda0745ecbc2ba5f99f664d6f/users/bdea9b919fc549bc9771606ab6e8a5af/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:02.985366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6545d-8f53-474a-a241-98bca01cb419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb17d85bda0745ecbc2ba5f99f664d6f, user_id=bdea9b919fc549bc9771606ab6e8a5af, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:02.988223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6545d-8f53-474a-a241-98bca01cb419 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:03.022516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:03.024221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:03.038188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaf0bb-457e-4aa6-bf1d-af8d41c89c52 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:03.042928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:03.043576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:03.043849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:03.044118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] PATH_INFO: `/v3/users/8266f0cadad84d17b92ecb8203e8fa1c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:03.045298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8266f0cadad84d17b92ecb8203e8fa1c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:03.047733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:03.118642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 949/1886] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:28:02 2022] PUT /identity/v3/projects/bb17d85bda0745ecbc2ba5f99f664d6f/users/bdea9b919fc549bc9771606ab6e8a5af/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:03.122676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6545d-8f53-474a-a241-98bca01cb419 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:03.192411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6545d-8f53-474a-a241-98bca01cb419 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:03.197795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762d64d-313e-4b34-80e6-c880515b0c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:03.199083 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762d64d-313e-4b34-80e6-c880515b0c6b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:03.199364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762d64d-313e-4b34-80e6-c880515b0c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:03.199640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762d64d-313e-4b34-80e6-c880515b0c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:03.200325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762d64d-313e-4b34-80e6-c880515b0c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:03.201985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762d64d-313e-4b34-80e6-c880515b0c6b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:03.213509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 950/1887] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:28:03.218919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762d64d-313e-4b34-80e6-c880515b0c6b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:03.237884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762d64d-313e-4b34-80e6-c880515b0c6b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:03.244248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a3f9b-5500-4fb3-98bd-49533d2ce3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:03.245195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a3f9b-5500-4fb3-98bd-49533d2ce3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:03.245702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a3f9b-5500-4fb3-98bd-49533d2ce3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:03.245977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a3f9b-5500-4fb3-98bd-49533d2ce3ca admin admin] PATH_INFO: `/v3/projects/bb17d85bda0745ecbc2ba5f99f664d6f/users/bdea9b919fc549bc9771606ab6e8a5af/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:03.247968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a3f9b-5500-4fb3-98bd-49533d2ce3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb17d85bda0745ecbc2ba5f99f664d6f, user_id=bdea9b919fc549bc9771606ab6e8a5af, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:03.251255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a3f9b-5500-4fb3-98bd-49533d2ce3ca admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:03.262403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 951/1888] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:28:03 2022] PUT /identity/v3/projects/bb17d85bda0745ecbc2ba5f99f664d6f/users/bdea9b919fc549bc9771606ab6e8a5af/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:28:03.361980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8266f0cadad84d17b92ecb8203e8fa1c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:03.369265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8266f0cadad84d17b92ecb8203e8fa1c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:03.381163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8266f0cadad84d17b92ecb8203e8fa1c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:03.472594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8266f0cadad84d17b92ecb8203e8fa1c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:03.473214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8266f0cadad84d17b92ecb8203e8fa1c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:03.487164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 938/1889] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:03 2022] DELETE /identity/v3/users/8266f0cadad84d17b92ecb8203e8fa1c => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:28:03.487271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0fec-f0af-49c6-a504-4fbcff6ab71f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:03.487271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0fec-f0af-49c6-a504-4fbcff6ab71f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:03.487271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0fec-f0af-49c6-a504-4fbcff6ab71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:03.505992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-85ec0fec-f0af-49c6-a504-4fbcff6ab71f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:03.507204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85ec0fec-f0af-49c6-a504-4fbcff6ab71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:03.574465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 952/1890] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:03.584666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:03.586333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:03.658263 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f42c1-5748-455a-a741-2c2e78acdbc7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:03.663561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922aabaa-4c39-4615-b1de-fc040ec1e8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:03.664439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aabaa-4c39-4615-b1de-fc040ec1e8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:03.664886 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aabaa-4c39-4615-b1de-fc040ec1e8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:03.665321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922aabaa-4c39-4615-b1de-fc040ec1e8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:03.680898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922aabaa-4c39-4615-b1de-fc040ec1e8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:03.682742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922aabaa-4c39-4615-b1de-fc040ec1e8d1 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:03.694382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 939/1891] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:03.707895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec0fec-f0af-49c6-a504-4fbcff6ab71f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:03.709574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-85ec0fec-f0af-49c6-a504-4fbcff6ab71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:03.723042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec0fec-f0af-49c6-a504-4fbcff6ab71f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:03.728020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:03.729034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:03.729414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:03.729762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] PATH_INFO: `/v3/users/fc35115b2d3d451aa13267a9d24b2e83` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:03.731139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc35115b2d3d451aa13267a9d24b2e83)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:03.733189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:03.752714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:03.957226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc35115b2d3d451aa13267a9d24b2e83'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:03.958071 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc35115b2d3d451aa13267a9d24b2e83'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.000489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc35115b2d3d451aa13267a9d24b2e83'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.009841 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc35115b2d3d451aa13267a9d24b2e83'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.017037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc35115b2d3d451aa13267a9d24b2e83'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.075534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 953/1892] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:03 2022] DELETE /identity/v3/users/fc35115b2d3d451aa13267a9d24b2e83 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:04.086494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cbcb9-3818-4bb3-81b5-338ba9601a8d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:04.086789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cbcb9-3818-4bb3-81b5-338ba9601a8d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:04.087043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cbcb9-3818-4bb3-81b5-338ba9601a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:04.122104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f63cbcb9-3818-4bb3-81b5-338ba9601a8d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:04.125044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f63cbcb9-3818-4bb3-81b5-338ba9601a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:04.134634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:04.240417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 940/1893] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:04.247396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:04.248253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:04.307691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cbcb9-3818-4bb3-81b5-338ba9601a8d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:04.308698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f63cbcb9-3818-4bb3-81b5-338ba9601a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:04.348227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cbcb9-3818-4bb3-81b5-338ba9601a8d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:04.367675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:04.367675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:04.367675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:04.367675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] PATH_INFO: `/v3/projects/7410f317be1b45ef98842e39affe5ded` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:04.369761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7410f317be1b45ef98842e39affe5ded)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:04.375310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:04.394186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cbaed-cc21-4712-828d-08b5bfad6b35 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:04.404541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e846e4-8b11-4b5e-8cd2-0a8f1a99afec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:04.405258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e846e4-8b11-4b5e-8cd2-0a8f1a99afec service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:04.405419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e846e4-8b11-4b5e-8cd2-0a8f1a99afec service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:04.405690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e846e4-8b11-4b5e-8cd2-0a8f1a99afec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:04.421666 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e846e4-8b11-4b5e-8cd2-0a8f1a99afec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:04.423643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e846e4-8b11-4b5e-8cd2-0a8f1a99afec service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:04.435144 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 954/1894] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:04.558606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7410f317be1b45ef98842e39affe5ded', 'request_id': 'req-27f83d7c-85c5-47ab-af49-a7d540858df6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.583370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7410f317be1b45ef98842e39affe5ded', 'request_id': 'req-27f83d7c-85c5-47ab-af49-a7d540858df6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.598981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7410f317be1b45ef98842e39affe5ded', 'request_id': 'req-27f83d7c-85c5-47ab-af49-a7d540858df6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.684364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] The token cache is being invalidate because project 7410f317be1b45ef98842e39affe5ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:04.684888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.685734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:04.688085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 941/1895] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:04 2022] DELETE /identity/v3/projects/7410f317be1b45ef98842e39affe5ded => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:28:04.692164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e846e4-8b11-4b5e-8cd2-0a8f1a99afec service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:04.703266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e846e4-8b11-4b5e-8cd2-0a8f1a99afec service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:04.766882 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:04.767553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:04.767850 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:04.768376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] PATH_INFO: `/v3/users/a93ff2ea73304b348290abe94cbc4ada` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:04.769239 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a93ff2ea73304b348290abe94cbc4ada)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:04.771185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:05.030919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:05.032555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:05.034203 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:05.057268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f83d7c-85c5-47ab-af49-a7d540858df6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:05.063579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a93ff2ea73304b348290abe94cbc4ada'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.065677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a93ff2ea73304b348290abe94cbc4ada'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.073235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:05.075702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:05.078275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:05.078810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] PATH_INFO: `/v3/projects/a4a17e6e1e7847fbace6d036669df116` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:05.088581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a17e6e1e7847fbace6d036669df116)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:05.104492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:05.121454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a93ff2ea73304b348290abe94cbc4ada'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.137834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a93ff2ea73304b348290abe94cbc4ada'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.139007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a93ff2ea73304b348290abe94cbc4ada'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.262272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 955/1896] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:04 2022] DELETE /identity/v3/users/a93ff2ea73304b348290abe94cbc4ada => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:28:05.281417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a17e6e1e7847fbace6d036669df116', 'request_id': 'req-a4aa9285-1734-41a5-8b11-e80a7d8633de'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.298424 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a17e6e1e7847fbace6d036669df116', 'request_id': 'req-a4aa9285-1734-41a5-8b11-e80a7d8633de'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.309761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a17e6e1e7847fbace6d036669df116', 'request_id': 'req-a4aa9285-1734-41a5-8b11-e80a7d8633de'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.433183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] The token cache is being invalidate because project a4a17e6e1e7847fbace6d036669df116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:05.433586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.434461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.439255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 942/1897] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:05 2022] DELETE /identity/v3/projects/a4a17e6e1e7847fbace6d036669df116 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:05.443235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:05.444800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:05.527741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e4b92-2bef-44ce-a7c4-ab3c6543a8a4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:05.670991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:05.673118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:05.679810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:05.679810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] PATH_INFO: `/v3/users/54e7743845f24065b49ffdd6373603d5` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:05.714904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e7743845f24065b49ffdd6373603d5)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:05.724949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:05.747481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:05.753806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:05.779975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa9285-1734-41a5-8b11-e80a7d8633de admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:05.799942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:05.800612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:05.803248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:05.804620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] PATH_INFO: `/v3/users/34fe65b1505d4c11991ebcf3a0cd4259` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:05.807839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34fe65b1505d4c11991ebcf3a0cd4259)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:05.819039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:05.842192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:05.979927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e7743845f24065b49ffdd6373603d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:05.981938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e7743845f24065b49ffdd6373603d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.007481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e7743845f24065b49ffdd6373603d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.017498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e7743845f24065b49ffdd6373603d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.018425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e7743845f24065b49ffdd6373603d5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.043099 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34fe65b1505d4c11991ebcf3a0cd4259'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.062525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34fe65b1505d4c11991ebcf3a0cd4259'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.080024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34fe65b1505d4c11991ebcf3a0cd4259'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.112578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 956/1898] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:05 2022] DELETE /identity/v3/users/54e7743845f24065b49ffdd6373603d5 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:06.113778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:06.115889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:06.204699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34fe65b1505d4c11991ebcf3a0cd4259'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.215685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e283e6be-6561-4c2b-9717-a073738ebed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34fe65b1505d4c11991ebcf3a0cd4259'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.215685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 943/1899] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:05 2022] DELETE /identity/v3/users/34fe65b1505d4c11991ebcf3a0cd4259 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:06.220685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76faaa4-d662-4730-aaeb-5314b6699a90 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:06.220971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76faaa4-d662-4730-aaeb-5314b6699a90 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:06.221298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76faaa4-d662-4730-aaeb-5314b6699a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:06.290456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d76faaa4-d662-4730-aaeb-5314b6699a90 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:06.290456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d76faaa4-d662-4730-aaeb-5314b6699a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:06.293066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db796b-beb2-48b6-a251-f79c8445d31b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:06.377462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:06.378512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:06.379645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:06.380469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] PATH_INFO: `/v3/projects/bbf51e400ebb4d5886ffb251ca48625d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:06.381937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf51e400ebb4d5886ffb251ca48625d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:06.407394 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:06.426366 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 944/1900] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:06.436473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76faaa4-d662-4730-aaeb-5314b6699a90 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:06.438116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d76faaa4-d662-4730-aaeb-5314b6699a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:06.617941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf51e400ebb4d5886ffb251ca48625d', 'request_id': 'req-fbf47945-5771-4abe-adbb-5f0885aa70b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.646152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76faaa4-d662-4730-aaeb-5314b6699a90 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:06.714634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:06.715511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:06.715967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:06.717403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:06.759944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf51e400ebb4d5886ffb251ca48625d', 'request_id': 'req-fbf47945-5771-4abe-adbb-5f0885aa70b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.777785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf51e400ebb4d5886ffb251ca48625d', 'request_id': 'req-fbf47945-5771-4abe-adbb-5f0885aa70b2'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.799812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] The token cache is being invalidate because project bbf51e400ebb4d5886ffb251ca48625d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:06.800475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.801365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:06.806463 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:06.807501 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 957/1901] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:06 2022] DELETE /identity/v3/projects/bbf51e400ebb4d5886ffb251ca48625d => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:06.811088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:06.813708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:06.842396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:06.846373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:06.861079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 945/1902] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:06.976250 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf47945-5771-4abe-adbb-5f0885aa70b2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:06.982981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:06.985149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:06.985149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:06.985343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] PATH_INFO: `/v3/users/71e02c187fd04e888ecf352908f8269c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:06.986563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e02c187fd04e888ecf352908f8269c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:06.989716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:07.033967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:07.036137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:07.066191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74221f76-b14f-4c88-a797-302e83890e33 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:07.074377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:07.077105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:07.078447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:07.079297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] PATH_INFO: `/v3/projects/a3883eba1ead4c0ca66b6119b9e2ae31` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:07.082874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3883eba1ead4c0ca66b6119b9e2ae31)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:07.088684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:07.212683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e02c187fd04e888ecf352908f8269c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.213437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e02c187fd04e888ecf352908f8269c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.235957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e02c187fd04e888ecf352908f8269c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.245955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e02c187fd04e888ecf352908f8269c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.246789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-c9fbfcd6-a6b6-486d-9335-83e25d4a4bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e02c187fd04e888ecf352908f8269c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.273080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3883eba1ead4c0ca66b6119b9e2ae31', 'request_id': 'req-0afd99af-3207-42d6-841a-4ef857b04cb4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.279746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3883eba1ead4c0ca66b6119b9e2ae31', 'request_id': 'req-0afd99af-3207-42d6-841a-4ef857b04cb4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.295419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3883eba1ead4c0ca66b6119b9e2ae31', 'request_id': 'req-0afd99af-3207-42d6-841a-4ef857b04cb4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.339272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 958/1903] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:06 2022] DELETE /identity/v3/users/71e02c187fd04e888ecf352908f8269c => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:07.398004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] The token cache is being invalidate because project a3883eba1ead4c0ca66b6119b9e2ae31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:07.403257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.408207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afd99af-3207-42d6-841a-4ef857b04cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:07.410146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 946/1904] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:07 2022] DELETE /identity/v3/projects/a3883eba1ead4c0ca66b6119b9e2ae31 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:07.417042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ddf32-8d20-4614-ad9a-6ea3476d7094 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:07.417675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ddf32-8d20-4614-ad9a-6ea3476d7094 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:07.418037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ddf32-8d20-4614-ad9a-6ea3476d7094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:07.464885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-114ddf32-8d20-4614-ad9a-6ea3476d7094 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:07.467646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-114ddf32-8d20-4614-ad9a-6ea3476d7094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:07.567711 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 959/1905] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:28:07.578069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93517b5d-f0a5-473d-ab1d-32aa955b76f8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:07.578602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93517b5d-f0a5-473d-ab1d-32aa955b76f8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:07.578889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93517b5d-f0a5-473d-ab1d-32aa955b76f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:07.661088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-93517b5d-f0a5-473d-ab1d-32aa955b76f8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:07.663415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-93517b5d-f0a5-473d-ab1d-32aa955b76f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:07.683842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 947/1906] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:07.688583 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ddf32-8d20-4614-ad9a-6ea3476d7094 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:07.703925 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ddf32-8d20-4614-ad9a-6ea3476d7094 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:07.708704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:07.709314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:07.709592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:07.709877 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:07.710543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:07.710842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:07.712714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:07.726305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 960/1907] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:07.730594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93517b5d-f0a5-473d-ab1d-32aa955b76f8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:07.758975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93517b5d-f0a5-473d-ab1d-32aa955b76f8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:07.773146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c7685-8968-4702-921d-97cb4b767b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:07.773821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c7685-8968-4702-921d-97cb4b767b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:07.774061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c7685-8968-4702-921d-97cb4b767b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:07.774505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c7685-8968-4702-921d-97cb4b767b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:07.775150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c7685-8968-4702-921d-97cb4b767b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:07.777083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c7685-8968-4702-921d-97cb4b767b53 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:07.821809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:07.823746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:07.842170 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710e1da-b3a9-4bf8-b294-5f589849dd40 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:07.850467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:07.851486 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:07.852069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:07.852775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] PATH_INFO: `/v3/projects/2eb16832d043491bbdf675ca6fb9f4cd` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:07.854954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb16832d043491bbdf675ca6fb9f4cd)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:07.866795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:07.877131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 948/1908] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:28:07 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:07.882785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c7685-8968-4702-921d-97cb4b767b53 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:07.891672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:07.965911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c7685-8968-4702-921d-97cb4b767b53 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:07.974161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:07.974783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:07.975065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:07.975336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:07.975985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:07.977976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:08.026491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb16832d043491bbdf675ca6fb9f4cd', 'request_id': 'req-76a170e3-f44b-43b6-9288-201703d3376f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.088595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb16832d043491bbdf675ca6fb9f4cd', 'request_id': 'req-76a170e3-f44b-43b6-9288-201703d3376f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.103119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 949/1909] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:28:07 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:08.104570 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:08.107890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb16832d043491bbdf675ca6fb9f4cd', 'request_id': 'req-76a170e3-f44b-43b6-9288-201703d3376f'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.110628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.128721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b05a4-f2d4-4c85-a0db-a94a25da2478 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:08.129378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] The token cache is being invalidate because project 2eb16832d043491bbdf675ca6fb9f4cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:08.129744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.130625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.133056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 961/1910] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:07 2022] DELETE /identity/v3/projects/2eb16832d043491bbdf675ca6fb9f4cd => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:08.134964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:08.136803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.193013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:08.193662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.194014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.194476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] PATH_INFO: `/v3/projects/034fd501ab084cf8acfd41c8fca50407` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.195425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=034fd501ab084cf8acfd41c8fca50407)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:08.197773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:08.216919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a170e3-f44b-43b6-9288-201703d3376f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:08.221983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c0b33-9d09-47a4-bfdb-9fe00e5867b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:08.222593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0b33-9d09-47a4-bfdb-9fe00e5867b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.222867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0b33-9d09-47a4-bfdb-9fe00e5867b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.223165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0b33-9d09-47a4-bfdb-9fe00e5867b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.223929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c0b33-9d09-47a4-bfdb-9fe00e5867b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:08.225797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c0b33-9d09-47a4-bfdb-9fe00e5867b2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:08.236743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 962/1911] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:08.237948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b11d-4d88-498a-b3b2-562eeceb6cb5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.238231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b11d-4d88-498a-b3b2-562eeceb6cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.238498 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73b11d-4d88-498a-b3b2-562eeceb6cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.260975 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5d73b11d-4d88-498a-b3b2-562eeceb6cb5 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:08.262267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5d73b11d-4d88-498a-b3b2-562eeceb6cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.272104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 963/1912] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:08.272104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73b11d-4d88-498a-b3b2-562eeceb6cb5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:08.283166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73b11d-4d88-498a-b3b2-562eeceb6cb5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:08.288919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787295cd-a4ea-4a14-9389-35c9ece523e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:08.289899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787295cd-a4ea-4a14-9389-35c9ece523e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.290550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787295cd-a4ea-4a14-9389-35c9ece523e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.291057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787295cd-a4ea-4a14-9389-35c9ece523e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.291984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787295cd-a4ea-4a14-9389-35c9ece523e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:08.293901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787295cd-a4ea-4a14-9389-35c9ece523e2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:08.307185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 964/1913] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:08.309300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434f7b2-9d7c-45e8-ab75-c9c14a7e0573 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.309779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434f7b2-9d7c-45e8-ab75-c9c14a7e0573 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.310348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434f7b2-9d7c-45e8-ab75-c9c14a7e0573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.333806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5434f7b2-9d7c-45e8-ab75-c9c14a7e0573 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:08.335824 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5434f7b2-9d7c-45e8-ab75-c9c14a7e0573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.345059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 965/1914] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:08.346249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434f7b2-9d7c-45e8-ab75-c9c14a7e0573 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:08.359214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434f7b2-9d7c-45e8-ab75-c9c14a7e0573 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:08.379905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '034fd501ab084cf8acfd41c8fca50407', 'request_id': 'req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.392012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '034fd501ab084cf8acfd41c8fca50407', 'request_id': 'req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.400964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '034fd501ab084cf8acfd41c8fca50407', 'request_id': 'req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.436487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62573-5dce-4a92-82f1-37e8e06323eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:08.437840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62573-5dce-4a92-82f1-37e8e06323eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.438129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62573-5dce-4a92-82f1-37e8e06323eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.438375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62573-5dce-4a92-82f1-37e8e06323eb admin admin] PATH_INFO: `/v3/projects/91d60e70bb7e4dd28d50c55ab12f228f/users/7e60760cb3c14c86a3290f584ee94ee4/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.456851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b62573-5dce-4a92-82f1-37e8e06323eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d60e70bb7e4dd28d50c55ab12f228f, user_id=7e60760cb3c14c86a3290f584ee94ee4, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:08.459771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b62573-5dce-4a92-82f1-37e8e06323eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:08.467138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] The token cache is being invalidate because project 034fd501ab084cf8acfd41c8fca50407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:08.467428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.468561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:08.472313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 950/1915] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:08 2022] DELETE /identity/v3/projects/034fd501ab084cf8acfd41c8fca50407 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:08.472313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:08.475052 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.493533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adefc5d-f65f-4aa0-9e8b-72a91fad3d44 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:08.499670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914cfb2-e945-4948-a7ce-c2e256e76ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:08.500180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914cfb2-e945-4948-a7ce-c2e256e76ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.500450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914cfb2-e945-4948-a7ce-c2e256e76ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.500764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914cfb2-e945-4948-a7ce-c2e256e76ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.501349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914cfb2-e945-4948-a7ce-c2e256e76ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:08.501625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914cfb2-e945-4948-a7ce-c2e256e76ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:08.503297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914cfb2-e945-4948-a7ce-c2e256e76ad7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:08.524647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 951/1916] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:08.524647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c2d2a-8ff5-44c1-b491-b219545fe3cc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.524647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c2d2a-8ff5-44c1-b491-b219545fe3cc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.524647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c2d2a-8ff5-44c1-b491-b219545fe3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.543583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-936c2d2a-8ff5-44c1-b491-b219545fe3cc None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:08.544870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-936c2d2a-8ff5-44c1-b491-b219545fe3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.553500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 966/1917] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:28:08 2022] PUT /identity/v3/projects/91d60e70bb7e4dd28d50c55ab12f228f/users/7e60760cb3c14c86a3290f584ee94ee4/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:08.556793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62573-5dce-4a92-82f1-37e8e06323eb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:08.628707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:08.641451 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 952/1918] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:08.645662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c26d52-3395-4aa8-af96-96dd5d2eacda None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.646473 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c26d52-3395-4aa8-af96-96dd5d2eacda None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.646760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c26d52-3395-4aa8-af96-96dd5d2eacda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.673308 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62573-5dce-4a92-82f1-37e8e06323eb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:08.675386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610267f-d52c-4f59-bbae-f5bd29ed6a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:08.676080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610267f-d52c-4f59-bbae-f5bd29ed6a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.676862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610267f-d52c-4f59-bbae-f5bd29ed6a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.676862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610267f-d52c-4f59-bbae-f5bd29ed6a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.677608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610267f-d52c-4f59-bbae-f5bd29ed6a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:08.679777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610267f-d52c-4f59-bbae-f5bd29ed6a75 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:08.718669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-72c26d52-3395-4aa8-af96-96dd5d2eacda None None] MFA Rules not processed for user `7e60760cb3c14c86a3290f584ee94ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:08.721250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-72c26d52-3395-4aa8-af96-96dd5d2eacda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.771778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 967/1919] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:28:08 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:08.772964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab236-2751-42da-99d2-f99d73d2ac3a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:08.773202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab236-2751-42da-99d2-f99d73d2ac3a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:08.773567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab236-2751-42da-99d2-f99d73d2ac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:08.808023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3d3ab236-2751-42da-99d2-f99d73d2ac3a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:08.808023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3d3ab236-2751-42da-99d2-f99d73d2ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.845753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 953/1920] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:08.847353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c26d52-3395-4aa8-af96-96dd5d2eacda None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:08.873998 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c26d52-3395-4aa8-af96-96dd5d2eacda None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:08.927660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 968/1921] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:08.928646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ab236-2751-42da-99d2-f99d73d2ac3a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:08.929950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3d3ab236-2751-42da-99d2-f99d73d2ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:08.999597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ab236-2751-42da-99d2-f99d73d2ac3a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.009207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c6e72-ebbc-4527-a6b2-5bb7a65933ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.009913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c6e72-ebbc-4527-a6b2-5bb7a65933ef service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.014019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c6e72-ebbc-4527-a6b2-5bb7a65933ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.014383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c6e72-ebbc-4527-a6b2-5bb7a65933ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.062293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d612519-e989-4a2d-bcf2-0ef39b319d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.062293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612519-e989-4a2d-bcf2-0ef39b319d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.062293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612519-e989-4a2d-bcf2-0ef39b319d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.062293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d612519-e989-4a2d-bcf2-0ef39b319d89 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.063023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d612519-e989-4a2d-bcf2-0ef39b319d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.063186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d612519-e989-4a2d-bcf2-0ef39b319d89 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.107349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:09.131644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c6e72-ebbc-4527-a6b2-5bb7a65933ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.131644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c6e72-ebbc-4527-a6b2-5bb7a65933ef service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.142635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 969/1922] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:09.143575 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c6e72-ebbc-4527-a6b2-5bb7a65933ef service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.154216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 954/1923] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:28:08 2022] POST /identity/v3/users => generated 288 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:09.159692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d612519-e989-4a2d-bcf2-0ef39b319d89 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.166995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c6e72-ebbc-4527-a6b2-5bb7a65933ef service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.179641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d612519-e989-4a2d-bcf2-0ef39b319d89 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.187351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.187351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.187351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.187351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.187752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:09.187752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.187752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.192882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a7be9-9795-4b07-acec-2b9ee1836bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.193583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a7be9-9795-4b07-acec-2b9ee1836bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.193839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a7be9-9795-4b07-acec-2b9ee1836bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.194024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a7be9-9795-4b07-acec-2b9ee1836bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.194670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a7be9-9795-4b07-acec-2b9ee1836bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.197620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a7be9-9795-4b07-acec-2b9ee1836bd3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.221295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 970/1924] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:09.230416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.233409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 955/1925] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:09.241304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a7be9-9795-4b07-acec-2b9ee1836bd3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.248080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de3690-75f6-4eb2-8bee-491ab6a9477a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.259921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a5324-0f98-41ad-b588-d6dee708ae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.260792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a5324-0f98-41ad-b588-d6dee708ae04 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.262076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a5324-0f98-41ad-b588-d6dee708ae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.262076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a5324-0f98-41ad-b588-d6dee708ae04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.262599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a5324-0f98-41ad-b588-d6dee708ae04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.265074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a5324-0f98-41ad-b588-d6dee708ae04 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.271336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a7be9-9795-4b07-acec-2b9ee1836bd3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.287421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41dee2-ce72-4b48-be7f-88aa1fab0753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.289545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41dee2-ce72-4b48-be7f-88aa1fab0753 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.289832 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41dee2-ce72-4b48-be7f-88aa1fab0753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.290111 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41dee2-ce72-4b48-be7f-88aa1fab0753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.290762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41dee2-ce72-4b48-be7f-88aa1fab0753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.298931 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41dee2-ce72-4b48-be7f-88aa1fab0753 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.320133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 956/1926] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:09.321120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41dee2-ce72-4b48-be7f-88aa1fab0753 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.334774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41dee2-ce72-4b48-be7f-88aa1fab0753 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.340477 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e07cc8-1777-4a87-867e-eaa82d60817c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.341142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e07cc8-1777-4a87-867e-eaa82d60817c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.341596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e07cc8-1777-4a87-867e-eaa82d60817c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.341880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e07cc8-1777-4a87-867e-eaa82d60817c admin admin] PATH_INFO: `/v3/projects/f9d530a42d8141d681c0684e0811aa01/users/ba05a3a036424007b4c5d08b639abddf/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.365738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e07cc8-1777-4a87-867e-eaa82d60817c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9d530a42d8141d681c0684e0811aa01, user_id=ba05a3a036424007b4c5d08b639abddf, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.369165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e07cc8-1777-4a87-867e-eaa82d60817c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.462716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 971/1927] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:28:09 2022] POST /identity/v3/projects => generated 373 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:09.466889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a5324-0f98-41ad-b588-d6dee708ae04 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.516495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 957/1928] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:28:09 2022] PUT /identity/v3/projects/f9d530a42d8141d681c0684e0811aa01/users/ba05a3a036424007b4c5d08b639abddf/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:09.518113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f47148-e5cb-42e9-ac33-f12ac598acbe None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.518469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f47148-e5cb-42e9-ac33-f12ac598acbe None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.518824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f47148-e5cb-42e9-ac33-f12ac598acbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.552829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-69f47148-e5cb-42e9-ac33-f12ac598acbe None None] MFA Rules not processed for user `ba05a3a036424007b4c5d08b639abddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:09.554723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-69f47148-e5cb-42e9-ac33-f12ac598acbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:09.559687 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a5324-0f98-41ad-b588-d6dee708ae04 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.715000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 958/1929] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:09.729651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453dd65-22e2-4b3b-870a-af7c3514a7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.729865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453dd65-22e2-4b3b-870a-af7c3514a7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.730028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453dd65-22e2-4b3b-870a-af7c3514a7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.730443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453dd65-22e2-4b3b-870a-af7c3514a7d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.731343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453dd65-22e2-4b3b-870a-af7c3514a7d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.734130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453dd65-22e2-4b3b-870a-af7c3514a7d2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.821623 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f47148-e5cb-42e9-ac33-f12ac598acbe None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.823215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-69f47148-e5cb-42e9-ac33-f12ac598acbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:09.880018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 972/1930] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:28:09 2022] POST /identity/v3/users => generated 287 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:28:09.884619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453dd65-22e2-4b3b-870a-af7c3514a7d2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.908592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453dd65-22e2-4b3b-870a-af7c3514a7d2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.924676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aef595-027b-4eb9-9521-12a5221d7ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.925854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aef595-027b-4eb9-9521-12a5221d7ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.926992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aef595-027b-4eb9-9521-12a5221d7ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.928917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aef595-027b-4eb9-9521-12a5221d7ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.930235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aef595-027b-4eb9-9521-12a5221d7ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.933652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aef595-027b-4eb9-9521-12a5221d7ea5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.960791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 973/1931] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:09.964487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f47148-e5cb-42e9-ac33-f12ac598acbe None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.964702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aef595-027b-4eb9-9521-12a5221d7ea5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:09.970144 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1e6ca-06d2-4aa3-9884-6f099c833d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.970842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1e6ca-06d2-4aa3-9884-6f099c833d56 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.971128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1e6ca-06d2-4aa3-9884-6f099c833d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.971404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1e6ca-06d2-4aa3-9884-6f099c833d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:09.983956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:09.988072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aef595-027b-4eb9-9521-12a5221d7ea5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:09.988990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1e6ca-06d2-4aa3-9884-6f099c833d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:09.990749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1e6ca-06d2-4aa3-9884-6f099c833d56 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:09.998243 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7df792-c80e-40cf-bb23-2749bb0a3522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:09.998980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7df792-c80e-40cf-bb23-2749bb0a3522 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:09.999688 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7df792-c80e-40cf-bb23-2749bb0a3522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:09.999974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7df792-c80e-40cf-bb23-2749bb0a3522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:10.000607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7df792-c80e-40cf-bb23-2749bb0a3522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:10.002182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7df792-c80e-40cf-bb23-2749bb0a3522 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:10.019077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 959/1932] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:28:10.026719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 974/1933] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:10.031231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1e6ca-06d2-4aa3-9884-6f099c833d56 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:10.054001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1e6ca-06d2-4aa3-9884-6f099c833d56 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:10.066787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:10.069064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:10.069706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:10.071017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] PATH_INFO: `/v3/projects/10316acf2adf4f34a63a08f0e60b2548/users/feb837dc77134d96be67f955683b2d7d/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:10.117995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10316acf2adf4f34a63a08f0e60b2548, user_id=feb837dc77134d96be67f955683b2d7d, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:10.120792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:10.287904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 960/1934] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:28:10 2022] PUT /identity/v3/projects/10316acf2adf4f34a63a08f0e60b2548/users/feb837dc77134d96be67f955683b2d7d/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:10.291020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c82b-7fc2-4bc1-bf3c-a27dbf90f0bf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:10.291020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c82b-7fc2-4bc1-bf3c-a27dbf90f0bf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:10.291020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c82b-7fc2-4bc1-bf3c-a27dbf90f0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:10.323858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2678c82b-7fc2-4bc1-bf3c-a27dbf90f0bf None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:10.325106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2678c82b-7fc2-4bc1-bf3c-a27dbf90f0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:10.426133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 975/1935] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:10.434991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:10.437272 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:10.538387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba790a-5ee3-44e0-a46e-4c57a8e38d09 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:10.543763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:10.544512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:10.544791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:10.545060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:10.563447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:10.563447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:10.571226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 961/1936] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:11.371376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b656d-6201-49d0-a123-6ba8f3f791ea None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:11.371898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b656d-6201-49d0-a123-6ba8f3f791ea None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:11.371898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b656d-6201-49d0-a123-6ba8f3f791ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:11.402685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2b0b656d-6201-49d0-a123-6ba8f3f791ea None None] MFA Rules not processed for user `bdea9b919fc549bc9771606ab6e8a5af`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:11.403926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2b0b656d-6201-49d0-a123-6ba8f3f791ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:11.405450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:11.523012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 976/1937] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:11.536056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:11.538236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:11.601315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aff374-2554-4b41-becb-0a46888264e5 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:11.603326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:11.603938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:11.604210 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:11.604513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:11.606264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:11.668301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:11.669748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:11.680352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 962/1938] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:12.461784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4a97-0e73-4796-862e-ce1f54ed1f50 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:12.462326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4a97-0e73-4796-862e-ce1f54ed1f50 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:12.462440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4a97-0e73-4796-862e-ce1f54ed1f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:12.488983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-755e4a97-0e73-4796-862e-ce1f54ed1f50 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:12.488983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-755e4a97-0e73-4796-862e-ce1f54ed1f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:12.566431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 977/1939] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:28:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:28:12.623273 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:12.626273 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:12.679746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:12.783444 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31a6f-c2f9-4f45-9451-0fa278eeeeee None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:12.783723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31a6f-c2f9-4f45-9451-0fa278eeeeee None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:12.784005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f31a6f-c2f9-4f45-9451-0fa278eeeeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:12.785554 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb355b-5cfe-49e7-8f81-e932da87473c service glance] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:12.801487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621f2fd-2f73-48ae-84c4-e9eb5205d385 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:12.802124 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621f2fd-2f73-48ae-84c4-e9eb5205d385 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:12.802403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621f2fd-2f73-48ae-84c4-e9eb5205d385 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:12.802665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621f2fd-2f73-48ae-84c4-e9eb5205d385 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:12.814540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621f2fd-2f73-48ae-84c4-e9eb5205d385 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:12.816138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621f2fd-2f73-48ae-84c4-e9eb5205d385 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:12.829762 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 963/1940] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:28:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:28:12.838756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d3f31a6f-c2f9-4f45-9451-0fa278eeeeee None None] MFA Rules not processed for user `feb837dc77134d96be67f955683b2d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:12.843400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d3f31a6f-c2f9-4f45-9451-0fa278eeeeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:12.956957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 978/1941] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:28:12.969622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621f2fd-2f73-48ae-84c4-e9eb5205d385 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:13.072961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621f2fd-2f73-48ae-84c4-e9eb5205d385 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:13.076868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f033de7-6f99-4d26-bc57-b2d7498a4e20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:13.083091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033de7-6f99-4d26-bc57-b2d7498a4e20 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:13.083091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033de7-6f99-4d26-bc57-b2d7498a4e20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:13.083091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033de7-6f99-4d26-bc57-b2d7498a4e20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:13.099438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f033de7-6f99-4d26-bc57-b2d7498a4e20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:13.102656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f033de7-6f99-4d26-bc57-b2d7498a4e20 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:13.121591 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 964/1942] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:13.872966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:14.003082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f31a6f-c2f9-4f45-9451-0fa278eeeeee None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:14.004663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d3f31a6f-c2f9-4f45-9451-0fa278eeeeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:14.096390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f31a6f-c2f9-4f45-9451-0fa278eeeeee None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:14.108912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:14.112848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:14.114220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:14.114534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] PATH_INFO: `/v3/users/7e60760cb3c14c86a3290f584ee94ee4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:14.118424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e60760cb3c14c86a3290f584ee94ee4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:14.122845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:14.394866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e60760cb3c14c86a3290f584ee94ee4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:14.395647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e60760cb3c14c86a3290f584ee94ee4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:14.424563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e60760cb3c14c86a3290f584ee94ee4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:14.439724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e60760cb3c14c86a3290f584ee94ee4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:14.440517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e60760cb3c14c86a3290f584ee94ee4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:14.499904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 979/1943] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:14 2022] DELETE /identity/v3/users/7e60760cb3c14c86a3290f584ee94ee4 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:14.505049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bf618-60df-43de-a868-a588f96e2f8d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:14.505643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bf618-60df-43de-a868-a588f96e2f8d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:14.506913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bf618-60df-43de-a868-a588f96e2f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:14.552763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:14.559703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-716bf618-60df-43de-a868-a588f96e2f8d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:14.563865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-716bf618-60df-43de-a868-a588f96e2f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:14.695459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 965/1944] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:14.706294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:14.708367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:14.835175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563cb2e-df02-4f70-a40f-db62501ea2d1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:14.840959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:14.841905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:14.842345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:14.844416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:14.857933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:14.860470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:14.900586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 980/1945] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:14.958980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:15.131939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bf618-60df-43de-a868-a588f96e2f8d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:15.132985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-716bf618-60df-43de-a868-a588f96e2f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:15.149970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bf618-60df-43de-a868-a588f96e2f8d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:15.165441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:15.166356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:15.166798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:15.167236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] PATH_INFO: `/v3/users/5c55e66cea624f21a7305de42b9a3948` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:15.168956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c55e66cea624f21a7305de42b9a3948)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:15.171423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:15.382270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c55e66cea624f21a7305de42b9a3948'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.391256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c55e66cea624f21a7305de42b9a3948'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.441699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c55e66cea624f21a7305de42b9a3948'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.456109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:15.467664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:15.486616 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049cab8-56fd-4523-9ab8-120d0ce128bd service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:15.493214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:15.494247 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:15.494584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:15.494799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] PATH_INFO: `/v3/projects/91d60e70bb7e4dd28d50c55ab12f228f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:15.495942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d60e70bb7e4dd28d50c55ab12f228f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:15.507610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:15.552715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c55e66cea624f21a7305de42b9a3948'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.553896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c55e66cea624f21a7305de42b9a3948'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.567213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 966/1946] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:15 2022] DELETE /identity/v3/users/5c55e66cea624f21a7305de42b9a3948 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:15.699611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d60e70bb7e4dd28d50c55ab12f228f', 'request_id': 'req-ed3f7668-b925-4b14-95a6-8937df6c976d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.753192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:15.815203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d60e70bb7e4dd28d50c55ab12f228f', 'request_id': 'req-ed3f7668-b925-4b14-95a6-8937df6c976d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.823773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d60e70bb7e4dd28d50c55ab12f228f', 'request_id': 'req-ed3f7668-b925-4b14-95a6-8937df6c976d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.844197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] The token cache is being invalidate because project 91d60e70bb7e4dd28d50c55ab12f228f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:15.844495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.845241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ed3f7668-b925-4b14-95a6-8937df6c976d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:15.846787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 981/1947] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:15 2022] DELETE /identity/v3/projects/91d60e70bb7e4dd28d50c55ab12f228f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:15.894502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:15.896087 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:16.022971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a385f5c-90d1-47ec-9ef9-6f4998fec306 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:16.135245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:16.135895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:16.136181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:16.136482 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] PATH_INFO: `/v3/projects/0e847aaaedd940f59efac66104ef8008` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:16.137604 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e847aaaedd940f59efac66104ef8008)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:16.140129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:16.300798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48367cb-8c2e-4b92-8639-d0a67fc47109 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:16.301156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48367cb-8c2e-4b92-8639-d0a67fc47109 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:16.301349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48367cb-8c2e-4b92-8639-d0a67fc47109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:16.323919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e847aaaedd940f59efac66104ef8008', 'request_id': 'req-1239634d-3589-48ce-beb6-7c39d24ff955'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:16.330674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c48367cb-8c2e-4b92-8639-d0a67fc47109 None None] MFA Rules not processed for user `feb837dc77134d96be67f955683b2d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:16.332116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c48367cb-8c2e-4b92-8639-d0a67fc47109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:16.351905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e847aaaedd940f59efac66104ef8008', 'request_id': 'req-1239634d-3589-48ce-beb6-7c39d24ff955'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:16.369220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:16.377525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e847aaaedd940f59efac66104ef8008', 'request_id': 'req-1239634d-3589-48ce-beb6-7c39d24ff955'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:16.422756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 982/1948] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:28:16.488895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] The token cache is being invalidate because project 0e847aaaedd940f59efac66104ef8008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:16.489196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:16.490945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:16.497348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 967/1949] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:15 2022] DELETE /identity/v3/projects/0e847aaaedd940f59efac66104ef8008 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:16.505854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48367cb-8c2e-4b92-8639-d0a67fc47109 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:16.505854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c48367cb-8c2e-4b92-8639-d0a67fc47109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:16.667394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:16.704746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48367cb-8c2e-4b92-8639-d0a67fc47109 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:16.719984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:16.721762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:16.722288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:16.722786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] PATH_INFO: `/v3/users/973564800262459cba25b09db74c87b3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:16.724584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=973564800262459cba25b09db74c87b3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:16.729853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:16.970697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:17.026134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973564800262459cba25b09db74c87b3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:17.028629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973564800262459cba25b09db74c87b3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:17.101229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973564800262459cba25b09db74c87b3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:17.120720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973564800262459cba25b09db74c87b3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:17.129832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-771f3c75-6654-4048-9270-684cf3e7d451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973564800262459cba25b09db74c87b3'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:17.216963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 983/1950] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:16 2022] DELETE /identity/v3/users/973564800262459cba25b09db74c87b3 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:17.324544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:17.583833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:17.586378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:17.614696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239634d-3589-48ce-beb6-7c39d24ff955 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:17.741857 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:17.742528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:17.742785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:17.743293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] PATH_INFO: `/v3/projects/b1ce17c89f044bb091ed819107f6d5ea` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:17.745334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ce17c89f044bb091ed819107f6d5ea)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:17.748091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:17.902514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ce17c89f044bb091ed819107f6d5ea', 'request_id': 'req-6b5efbe7-7393-41aa-b266-570e3a6a6021'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:17.923810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ce17c89f044bb091ed819107f6d5ea', 'request_id': 'req-6b5efbe7-7393-41aa-b266-570e3a6a6021'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:17.930617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ce17c89f044bb091ed819107f6d5ea', 'request_id': 'req-6b5efbe7-7393-41aa-b266-570e3a6a6021'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:18.478105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] The token cache is being invalidate because project b1ce17c89f044bb091ed819107f6d5ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:18.484298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:18.484298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6b5efbe7-7393-41aa-b266-570e3a6a6021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:18.484298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 968/1951] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:17 2022] DELETE /identity/v3/projects/b1ce17c89f044bb091ed819107f6d5ea => generated 0 bytes in 897 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:18.485658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced53193-300c-4b42-b656-343e821f9f5a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:18.486157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced53193-300c-4b42-b656-343e821f9f5a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:18.486244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced53193-300c-4b42-b656-343e821f9f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:18.512947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ced53193-300c-4b42-b656-343e821f9f5a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:18.514215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ced53193-300c-4b42-b656-343e821f9f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:18.594330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 984/1952] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:28:18.603445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ce5d5-5580-4a1f-bad7-2f689da8ee20 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:18.604024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ce5d5-5580-4a1f-bad7-2f689da8ee20 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:18.604024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ce5d5-5580-4a1f-bad7-2f689da8ee20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:18.671492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-793ce5d5-5580-4a1f-bad7-2f689da8ee20 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:18.674942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-793ce5d5-5580-4a1f-bad7-2f689da8ee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:18.688872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 969/1953] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:18.693435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced53193-300c-4b42-b656-343e821f9f5a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:18.707438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced53193-300c-4b42-b656-343e821f9f5a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:18.712625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:18.713297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:18.713718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:18.714147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:18.715077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:18.715364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:18.723215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:18.749434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 985/1954] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:18.753160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ce5d5-5580-4a1f-bad7-2f689da8ee20 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:18.771438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:18.782811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ce5d5-5580-4a1f-bad7-2f689da8ee20 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:18.788977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b97743-d6ee-4d0c-bef0-2ba469263df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:18.789601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b97743-d6ee-4d0c-bef0-2ba469263df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:18.790556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b97743-d6ee-4d0c-bef0-2ba469263df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:18.790988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b97743-d6ee-4d0c-bef0-2ba469263df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:18.791869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b97743-d6ee-4d0c-bef0-2ba469263df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:18.793995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b97743-d6ee-4d0c-bef0-2ba469263df0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:18.911475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 970/1955] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:28:18 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:18.915625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:18.928552 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619a1bf-f5c3-4ce2-8cc4-c7bd3f946d06 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:18.935499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:19.003461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b9197-ea72-4ae3-ac2f-b892c9eb855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:19.004156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b9197-ea72-4ae3-ac2f-b892c9eb855c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:19.004400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b9197-ea72-4ae3-ac2f-b892c9eb855c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:19.004656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b9197-ea72-4ae3-ac2f-b892c9eb855c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:19.005418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b9197-ea72-4ae3-ac2f-b892c9eb855c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:19.007342 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b9197-ea72-4ae3-ac2f-b892c9eb855c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:19.156581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 986/1956] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:28:18 2022] POST /identity/v3/users => generated 288 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:19.162834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b97743-d6ee-4d0c-bef0-2ba469263df0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:19.177389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b97743-d6ee-4d0c-bef0-2ba469263df0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:19.182060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3c226-2281-4cfe-a1ca-592dc3bc1027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:19.182790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3c226-2281-4cfe-a1ca-592dc3bc1027 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:19.183170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3c226-2281-4cfe-a1ca-592dc3bc1027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:19.183541 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3c226-2281-4cfe-a1ca-592dc3bc1027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:19.184313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3c226-2281-4cfe-a1ca-592dc3bc1027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:19.186615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3c226-2281-4cfe-a1ca-592dc3bc1027 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:19.203833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 971/1957] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:19.207882 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b9197-ea72-4ae3-ac2f-b892c9eb855c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:19.230996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b9197-ea72-4ae3-ac2f-b892c9eb855c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:19.248218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ed944-785c-437b-a6b5-73f52a00d91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:19.255418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ed944-785c-437b-a6b5-73f52a00d91e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:19.255418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ed944-785c-437b-a6b5-73f52a00d91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:19.255418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ed944-785c-437b-a6b5-73f52a00d91e admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:19.255418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ed944-785c-437b-a6b5-73f52a00d91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:19.255418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ed944-785c-437b-a6b5-73f52a00d91e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:19.270137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 987/1958] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:28:19.274017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3c226-2281-4cfe-a1ca-592dc3bc1027 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:19.285504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3c226-2281-4cfe-a1ca-592dc3bc1027 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:19.291859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:19.292605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:19.292890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:19.293164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] PATH_INFO: `/v3/projects/78ff5bdf6d8a48abb520cc27c8fe3ad2/users/0eab083b1ef94dffb849d7e59cec0488/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:19.313309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ff5bdf6d8a48abb520cc27c8fe3ad2, user_id=0eab083b1ef94dffb849d7e59cec0488, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:19.316435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:19.403050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 972/1959] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:28:19 2022] PUT /identity/v3/projects/78ff5bdf6d8a48abb520cc27c8fe3ad2/users/0eab083b1ef94dffb849d7e59cec0488/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:19.404976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f9a40-c7f5-4af4-a136-d1b28ccd0c5b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:19.405268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f9a40-c7f5-4af4-a136-d1b28ccd0c5b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:19.405552 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f9a40-c7f5-4af4-a136-d1b28ccd0c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:19.467214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-789f9a40-c7f5-4af4-a136-d1b28ccd0c5b None None] MFA Rules not processed for user `0eab083b1ef94dffb849d7e59cec0488`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:19.468777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-789f9a40-c7f5-4af4-a136-d1b28ccd0c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:19.580415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 988/1960] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:28:19.586927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:19.590161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:19.592116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:19.680490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fc544-c8ba-4d92-a636-3c4c3a842f14 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:19.692328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:19.692328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:19.692328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:19.692328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:19.715072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:19.718726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:19.735456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 973/1961] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:22.065719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f9f5-f94b-48ad-8cde-307a879b276c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:22.066184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f9f5-f94b-48ad-8cde-307a879b276c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:22.068456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f9f5-f94b-48ad-8cde-307a879b276c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:22.099616 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7b57f9f5-f94b-48ad-8cde-307a879b276c None None] MFA Rules not processed for user `633c6c68e15f4be9a7289d15cac9de43`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:22.102045 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7b57f9f5-f94b-48ad-8cde-307a879b276c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:22.197702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 989/1962] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:27.788789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:27.790874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:27.881508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb641a7-b683-41ef-a29d-932c3116e54f service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:27.887579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:27.888337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:27.888623 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:27.889423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:27.891209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:27.904529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:27.907991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:27.922233 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 974/1963] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:28.154525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67327eb-7aa3-47de-a8cc-e9ff79d79c86 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:28.155161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67327eb-7aa3-47de-a8cc-e9ff79d79c86 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:28.155272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67327eb-7aa3-47de-a8cc-e9ff79d79c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:28.179338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a67327eb-7aa3-47de-a8cc-e9ff79d79c86 None None] MFA Rules not processed for user `834b92eb61864d1b95e9f037379c2e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:28.184876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a67327eb-7aa3-47de-a8cc-e9ff79d79c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:28.259771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 990/1964] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:35.499949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:35.501394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:35.542930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:35.566836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4122d-a316-4a6d-8d67-2afe911f892a service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:35.570891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:35.571518 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:35.571811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:35.572067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] PATH_INFO: `/v3/users/ba05a3a036424007b4c5d08b639abddf` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:35.573281 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba05a3a036424007b4c5d08b639abddf)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:35.575177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:35.757778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba05a3a036424007b4c5d08b639abddf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:35.765609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba05a3a036424007b4c5d08b639abddf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:35.777846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba05a3a036424007b4c5d08b639abddf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:35.854154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba05a3a036424007b4c5d08b639abddf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:35.854861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba05a3a036424007b4c5d08b639abddf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:35.862327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 975/1965] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:35 2022] DELETE /identity/v3/users/ba05a3a036424007b4c5d08b639abddf => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:35.866602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191e295-f706-40ab-b092-196accd2d082 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:35.870268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191e295-f706-40ab-b092-196accd2d082 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:35.870268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191e295-f706-40ab-b092-196accd2d082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:35.886081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f191e295-f706-40ab-b092-196accd2d082 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:35.887287 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f191e295-f706-40ab-b092-196accd2d082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:35.946912 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 991/1966] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:35.957318 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:35.962080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:36.018109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd9c57-81cd-44e8-b562-9b88b0c0e18e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:36.021921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917d428-a510-4e7e-978a-c28f80ff2290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:36.022538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917d428-a510-4e7e-978a-c28f80ff2290 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:36.022816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917d428-a510-4e7e-978a-c28f80ff2290 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:36.023089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917d428-a510-4e7e-978a-c28f80ff2290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:36.033456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917d428-a510-4e7e-978a-c28f80ff2290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:36.034887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917d428-a510-4e7e-978a-c28f80ff2290 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:36.044360 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 976/1967] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:36.478761 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191e295-f706-40ab-b092-196accd2d082 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:36.480340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f191e295-f706-40ab-b092-196accd2d082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:36.494174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191e295-f706-40ab-b092-196accd2d082 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:36.499182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:36.499842 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:36.500154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:36.500506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] PATH_INFO: `/v3/projects/f9d530a42d8141d681c0684e0811aa01` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:36.501574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9d530a42d8141d681c0684e0811aa01)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:36.503475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:36.597200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:36.599991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9d530a42d8141d681c0684e0811aa01', 'request_id': 'req-e3922b23-dfd3-4840-801d-24f7d5d02ea1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:36.665920 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9d530a42d8141d681c0684e0811aa01', 'request_id': 'req-e3922b23-dfd3-4840-801d-24f7d5d02ea1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:36.673737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9d530a42d8141d681c0684e0811aa01', 'request_id': 'req-e3922b23-dfd3-4840-801d-24f7d5d02ea1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:36.685630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] The token cache is being invalidate because project f9d530a42d8141d681c0684e0811aa01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:36.686628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:36.687349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:36.688946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 992/1968] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:36 2022] DELETE /identity/v3/projects/f9d530a42d8141d681c0684e0811aa01 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:38.637398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f223880-f7c7-4e13-9c8f-c98514138cbf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:38.637398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f223880-f7c7-4e13-9c8f-c98514138cbf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:38.637398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f223880-f7c7-4e13-9c8f-c98514138cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:38.657711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5f223880-f7c7-4e13-9c8f-c98514138cbf None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:38.658912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5f223880-f7c7-4e13-9c8f-c98514138cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:38.740500 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 977/1969] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:28:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:38.752813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:38.755298 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:38.850806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3922b23-dfd3-4840-801d-24f7d5d02ea1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:38.856428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991dd7a-0d1b-4652-844b-b289d5644a94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:38.857173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991dd7a-0d1b-4652-844b-b289d5644a94 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:38.857543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991dd7a-0d1b-4652-844b-b289d5644a94 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:38.858840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991dd7a-0d1b-4652-844b-b289d5644a94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:38.867338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991dd7a-0d1b-4652-844b-b289d5644a94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:38.869101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991dd7a-0d1b-4652-844b-b289d5644a94 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:38.877504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 993/1970] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:28:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:28:48.729876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:48.874458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:54.227340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f223880-f7c7-4e13-9c8f-c98514138cbf None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:54.228805 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5f223880-f7c7-4e13-9c8f-c98514138cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:54.292302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f223880-f7c7-4e13-9c8f-c98514138cbf None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:54.297106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:54.297730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.298010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.298279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] PATH_INFO: `/v3/users/0eab083b1ef94dffb849d7e59cec0488` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.299441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eab083b1ef94dffb849d7e59cec0488)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:54.301304 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:54.433740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eab083b1ef94dffb849d7e59cec0488'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:54.441689 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eab083b1ef94dffb849d7e59cec0488'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:54.454836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eab083b1ef94dffb849d7e59cec0488'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:54.516445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eab083b1ef94dffb849d7e59cec0488'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:54.517056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eab083b1ef94dffb849d7e59cec0488'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:54.523173 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 978/1971] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:28:54 2022] DELETE /identity/v3/users/0eab083b1ef94dffb849d7e59cec0488 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:54.527778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48366-d000-44ff-9604-4766ff2a6492 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.528313 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48366-d000-44ff-9604-4766ff2a6492 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.528415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48366-d000-44ff-9604-4766ff2a6492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.547084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d4b48366-d000-44ff-9604-4766ff2a6492 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:54.548295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d4b48366-d000-44ff-9604-4766ff2a6492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:54.606985 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 994/1972] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:54.616592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:54.618010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:54.675966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8075a6a-9717-46f6-8fa8-8b0e89e1f0d7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:54.679713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912b7da-10a4-4ea3-b51a-901cb9dc2f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:54.680348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b7da-10a4-4ea3-b51a-901cb9dc2f64 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:54.680756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b7da-10a4-4ea3-b51a-901cb9dc2f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:54.680882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b7da-10a4-4ea3-b51a-901cb9dc2f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:54.690920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912b7da-10a4-4ea3-b51a-901cb9dc2f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:54.692415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912b7da-10a4-4ea3-b51a-901cb9dc2f64 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:54.702321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 979/1973] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:28:55.119759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b48366-d000-44ff-9604-4766ff2a6492 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.120166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d4b48366-d000-44ff-9604-4766ff2a6492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:55.133664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b48366-d000-44ff-9604-4766ff2a6492 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.138513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.139175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.139459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.139732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] PATH_INFO: `/v3/projects/78ff5bdf6d8a48abb520cc27c8fe3ad2` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.140873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ff5bdf6d8a48abb520cc27c8fe3ad2)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.142773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.248554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ff5bdf6d8a48abb520cc27c8fe3ad2', 'request_id': 'req-d7c50568-4884-4c17-8ee0-4a04ac376549'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:55.293586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ff5bdf6d8a48abb520cc27c8fe3ad2', 'request_id': 'req-d7c50568-4884-4c17-8ee0-4a04ac376549'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:55.300660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ff5bdf6d8a48abb520cc27c8fe3ad2', 'request_id': 'req-d7c50568-4884-4c17-8ee0-4a04ac376549'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:55.311014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] The token cache is being invalidate because project 78ff5bdf6d8a48abb520cc27c8fe3ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:28:55.311300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:55.311928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-d7c50568-4884-4c17-8ee0-4a04ac376549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:28:55.313501 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 995/1974] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:28:55 2022] DELETE /identity/v3/projects/78ff5bdf6d8a48abb520cc27c8fe3ad2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:55.318365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eb0aa-da76-4148-a66c-ebd8429333c3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.318849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eb0aa-da76-4148-a66c-ebd8429333c3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.318969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eb0aa-da76-4148-a66c-ebd8429333c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.338270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9f5eb0aa-da76-4148-a66c-ebd8429333c3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:55.339434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9f5eb0aa-da76-4148-a66c-ebd8429333c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:55.356453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:55.398313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 980/1975] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:55.407516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f384b54-6449-4903-bad9-345d47e5b1ee None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.407801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f384b54-6449-4903-bad9-345d47e5b1ee None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.408072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f384b54-6449-4903-bad9-345d47e5b1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.428215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2f384b54-6449-4903-bad9-345d47e5b1ee None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:55.429459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2f384b54-6449-4903-bad9-345d47e5b1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:55.437408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 996/1976] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:55.441418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5eb0aa-da76-4148-a66c-ebd8429333c3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.450788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5eb0aa-da76-4148-a66c-ebd8429333c3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.456047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.456674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.456949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.457218 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.457853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:28:55.458154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.459743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.477084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 981/1977] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:28:55.484647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f384b54-6449-4903-bad9-345d47e5b1ee None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.495133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f384b54-6449-4903-bad9-345d47e5b1ee None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.500213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f406d-66a8-4810-a933-c9c02e824b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.500915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f406d-66a8-4810-a933-c9c02e824b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.501297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f406d-66a8-4810-a933-c9c02e824b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.502321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f406d-66a8-4810-a933-c9c02e824b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.502321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f406d-66a8-4810-a933-c9c02e824b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.504540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f406d-66a8-4810-a933-c9c02e824b7f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.604544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 997/1978] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:28:55 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:28:55.605966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.617296 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e619fad-ebda-49bd-b3cd-803059089ea8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.676512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5166d-0c50-4b1f-b2ec-a5792d9a758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.677114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5166d-0c50-4b1f-b2ec-a5792d9a758b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.677394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5166d-0c50-4b1f-b2ec-a5792d9a758b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.677666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5166d-0c50-4b1f-b2ec-a5792d9a758b admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.678297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5166d-0c50-4b1f-b2ec-a5792d9a758b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.680207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5166d-0c50-4b1f-b2ec-a5792d9a758b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.785702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 982/1979] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:28:55 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:28:55.789899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f406d-66a8-4810-a933-c9c02e824b7f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.800214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f406d-66a8-4810-a933-c9c02e824b7f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.804860 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a407bf9-3574-478c-a604-76cd0172d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.805457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a407bf9-3574-478c-a604-76cd0172d573 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.805733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a407bf9-3574-478c-a604-76cd0172d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.806032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a407bf9-3574-478c-a604-76cd0172d573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.806668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a407bf9-3574-478c-a604-76cd0172d573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.808252 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a407bf9-3574-478c-a604-76cd0172d573 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.818677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 998/1980] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:55.822981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5166d-0c50-4b1f-b2ec-a5792d9a758b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.833389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5166d-0c50-4b1f-b2ec-a5792d9a758b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.838004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec1570-223b-4d4a-99c2-26fd2345da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.838635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec1570-223b-4d4a-99c2-26fd2345da53 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.838888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec1570-223b-4d4a-99c2-26fd2345da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.839235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec1570-223b-4d4a-99c2-26fd2345da53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.839773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec1570-223b-4d4a-99c2-26fd2345da53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.841419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec1570-223b-4d4a-99c2-26fd2345da53 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.852130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 983/1981] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:28:55.856182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a407bf9-3574-478c-a604-76cd0172d573 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:55.866157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a407bf9-3574-478c-a604-76cd0172d573 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:55.870680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:55.871319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.871596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.871865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] PATH_INFO: `/v3/projects/a9111fe68cb848a8913a7891db7e9c50/users/d8177c0225b14755bf9593e094a288c9/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:55.887508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9111fe68cb848a8913a7891db7e9c50, user_id=d8177c0225b14755bf9593e094a288c9, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:55.890845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:55.957943 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 999/1982] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:28:55 2022] PUT /identity/v3/projects/a9111fe68cb848a8913a7891db7e9c50/users/d8177c0225b14755bf9593e094a288c9/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:28:55.979637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74383587-6f34-413a-90aa-988c083849fb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:55.979985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74383587-6f34-413a-90aa-988c083849fb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:55.980461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74383587-6f34-413a-90aa-988c083849fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:56.010606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-74383587-6f34-413a-90aa-988c083849fb None None] MFA Rules not processed for user `d8177c0225b14755bf9593e094a288c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:28:56.011823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-74383587-6f34-413a-90aa-988c083849fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:56.081310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 984/1983] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:28:56.094660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:28:56.095857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:28:56.121605 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:28:56.159812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aa322-a5d7-40ec-893f-2631fe06b2c4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:28:56.164477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc805d-adcd-4d9e-bf08-41bfa9d0a5bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:28:56.165178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc805d-adcd-4d9e-bf08-41bfa9d0a5bf service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:28:56.165453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc805d-adcd-4d9e-bf08-41bfa9d0a5bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:28:56.165720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc805d-adcd-4d9e-bf08-41bfa9d0a5bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:28:56.174858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc805d-adcd-4d9e-bf08-41bfa9d0a5bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:28:56.180419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc805d-adcd-4d9e-bf08-41bfa9d0a5bf service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:28:56.185326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1000/1984] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:28:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:29:01.058276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74383587-6f34-413a-90aa-988c083849fb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:01.078438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:01.124046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74383587-6f34-413a-90aa-988c083849fb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:01.128727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:01.129353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:01.129626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:01.129920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] PATH_INFO: `/v3/users/d8177c0225b14755bf9593e094a288c9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:01.131012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8177c0225b14755bf9593e094a288c9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:01.133057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:01.347178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8177c0225b14755bf9593e094a288c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:01.354058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8177c0225b14755bf9593e094a288c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:01.368055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8177c0225b14755bf9593e094a288c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:01.430231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8177c0225b14755bf9593e094a288c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:01.430829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8177c0225b14755bf9593e094a288c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:01.436924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 985/1985] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:29:01 2022] DELETE /identity/v3/users/d8177c0225b14755bf9593e094a288c9 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:01.441400 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691c207-3226-4c1b-9510-5b55b9deeb8a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:01.441861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691c207-3226-4c1b-9510-5b55b9deeb8a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:01.441975 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691c207-3226-4c1b-9510-5b55b9deeb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:01.461210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f691c207-3226-4c1b-9510-5b55b9deeb8a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:01.462395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f691c207-3226-4c1b-9510-5b55b9deeb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:01.519096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1001/1986] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:29:01.528801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:01.530869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:01.586334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1bd4f-1415-4280-bac0-574fe899cf9a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:01.590068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32241efe-e2dc-4b7c-9bdf-ac1a846271f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:01.590693 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241efe-e2dc-4b7c-9bdf-ac1a846271f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:01.590968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241efe-e2dc-4b7c-9bdf-ac1a846271f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:01.591237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241efe-e2dc-4b7c-9bdf-ac1a846271f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:01.600773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32241efe-e2dc-4b7c-9bdf-ac1a846271f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:01.602220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32241efe-e2dc-4b7c-9bdf-ac1a846271f6 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:01.611438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 986/1987] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:29:02.075915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691c207-3226-4c1b-9510-5b55b9deeb8a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.077944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f691c207-3226-4c1b-9510-5b55b9deeb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:02.093362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691c207-3226-4c1b-9510-5b55b9deeb8a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.094674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.101792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.101792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.101792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] PATH_INFO: `/v3/projects/a9111fe68cb848a8913a7891db7e9c50` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.101792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9111fe68cb848a8913a7891db7e9c50)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.101792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.244036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9111fe68cb848a8913a7891db7e9c50', 'request_id': 'req-9519b83c-c346-4125-87e2-f16123b7f59d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:02.328568 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9111fe68cb848a8913a7891db7e9c50', 'request_id': 'req-9519b83c-c346-4125-87e2-f16123b7f59d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:02.336034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9111fe68cb848a8913a7891db7e9c50', 'request_id': 'req-9519b83c-c346-4125-87e2-f16123b7f59d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:02.347977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] The token cache is being invalidate because project a9111fe68cb848a8913a7891db7e9c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:29:02.348205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:02.348938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9519b83c-c346-4125-87e2-f16123b7f59d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:29:02.350599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1002/1988] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:29:02 2022] DELETE /identity/v3/projects/a9111fe68cb848a8913a7891db7e9c50 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:02.355422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639bea7-d9e4-4572-a4d4-d36c39c07625 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.355894 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639bea7-d9e4-4572-a4d4-d36c39c07625 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.356007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639bea7-d9e4-4572-a4d4-d36c39c07625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.376904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a639bea7-d9e4-4572-a4d4-d36c39c07625 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:02.378153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a639bea7-d9e4-4572-a4d4-d36c39c07625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:02.413213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:02.439576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 987/1989] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:29:02.447490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235618d-7d79-443d-a153-7b44d45801d2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.448013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235618d-7d79-443d-a153-7b44d45801d2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.448013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235618d-7d79-443d-a153-7b44d45801d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.467939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4235618d-7d79-443d-a153-7b44d45801d2 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:02.469160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4235618d-7d79-443d-a153-7b44d45801d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:02.476982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1003/1990] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:29:02.481113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639bea7-d9e4-4572-a4d4-d36c39c07625 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.491671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639bea7-d9e4-4572-a4d4-d36c39c07625 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.495780 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.496400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.496676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.496948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.497576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:29:02.497877 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.500146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.511468 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 988/1991] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:29:02.515603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235618d-7d79-443d-a153-7b44d45801d2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.524628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235618d-7d79-443d-a153-7b44d45801d2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.529136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93096efe-0387-41f4-b0ce-af7e12450ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.530248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93096efe-0387-41f4-b0ce-af7e12450ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.530522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93096efe-0387-41f4-b0ce-af7e12450ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.530791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93096efe-0387-41f4-b0ce-af7e12450ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.531397 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93096efe-0387-41f4-b0ce-af7e12450ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.533314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93096efe-0387-41f4-b0ce-af7e12450ef4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.614924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1004/1992] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:29:02 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:02.618935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.628881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ed84c-d2f4-48e1-9fdb-9bb90864d6b9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.684093 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da781c-cef9-41bc-8d0d-f61dd6a38fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.684717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da781c-cef9-41bc-8d0d-f61dd6a38fbf admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.684992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da781c-cef9-41bc-8d0d-f61dd6a38fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.685261 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da781c-cef9-41bc-8d0d-f61dd6a38fbf admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.685870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da781c-cef9-41bc-8d0d-f61dd6a38fbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.687768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da781c-cef9-41bc-8d0d-f61dd6a38fbf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.761371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 989/1993] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:29:02 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:29:02.765466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93096efe-0387-41f4-b0ce-af7e12450ef4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.775630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93096efe-0387-41f4-b0ce-af7e12450ef4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.780675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98760ecd-b945-4f7d-88a5-afc1859cbe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.781256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98760ecd-b945-4f7d-88a5-afc1859cbe4f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.781530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98760ecd-b945-4f7d-88a5-afc1859cbe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.781797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98760ecd-b945-4f7d-88a5-afc1859cbe4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.782423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98760ecd-b945-4f7d-88a5-afc1859cbe4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.783986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98760ecd-b945-4f7d-88a5-afc1859cbe4f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.793631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1005/1994] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:29:02.798159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da781c-cef9-41bc-8d0d-f61dd6a38fbf admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.807917 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da781c-cef9-41bc-8d0d-f61dd6a38fbf admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.812609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40242b9-13c8-4138-bbf1-87d34570297c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.813189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40242b9-13c8-4138-bbf1-87d34570297c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.813464 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40242b9-13c8-4138-bbf1-87d34570297c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.813732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40242b9-13c8-4138-bbf1-87d34570297c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.814355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40242b9-13c8-4138-bbf1-87d34570297c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.815940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40242b9-13c8-4138-bbf1-87d34570297c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.828038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 990/1995] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:29:02.831969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98760ecd-b945-4f7d-88a5-afc1859cbe4f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:02.842188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98760ecd-b945-4f7d-88a5-afc1859cbe4f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:02.846653 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:02.847282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.847555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.847823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] PATH_INFO: `/v3/projects/ca3097281d4c43458b408414648233a0/users/315f9ad503b8402090faac7b827b5463/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.849974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:02.862917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3097281d4c43458b408414648233a0, user_id=315f9ad503b8402090faac7b827b5463, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:02.865738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:02.927363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1006/1996] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:29:02 2022] PUT /identity/v3/projects/ca3097281d4c43458b408414648233a0/users/315f9ad503b8402090faac7b827b5463/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:29:02.932339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121be091-ad71-4ab5-8ef2-416d7507cd3c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:02.932626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121be091-ad71-4ab5-8ef2-416d7507cd3c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:02.932903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121be091-ad71-4ab5-8ef2-416d7507cd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:02.952129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-121be091-ad71-4ab5-8ef2-416d7507cd3c None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:02.953356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-121be091-ad71-4ab5-8ef2-416d7507cd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:03.011935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 991/1997] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:29:03.022289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:03.024033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:03.084088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77179a1-a794-4cdb-afe4-eba8e89503e8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:03.101922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:03.102549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:03.102828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:03.103098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:03.113374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:03.114805 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:03.130835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1007/1998] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:29:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:29:04.209986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9c27-02af-4423-8219-3218a2afbd03 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:04.210519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9c27-02af-4423-8219-3218a2afbd03 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:04.211223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9c27-02af-4423-8219-3218a2afbd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:04.256847 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1d7d9c27-02af-4423-8219-3218a2afbd03 None None] MFA Rules not processed for user `315f9ad503b8402090faac7b827b5463`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:04.261275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1d7d9c27-02af-4423-8219-3218a2afbd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:04.312196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:04.362627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 992/1999] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:29:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:29:04.375629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:04.377222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:04.438395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc3390-0b2b-4b5a-a235-42e94ff705ca service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:04.441946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce6b9d-8778-4e95-9a46-11571128a505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:04.442652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce6b9d-8778-4e95-9a46-11571128a505 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:04.442929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce6b9d-8778-4e95-9a46-11571128a505 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:04.443201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce6b9d-8778-4e95-9a46-11571128a505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:04.452104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce6b9d-8778-4e95-9a46-11571128a505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:04.453497 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce6b9d-8778-4e95-9a46-11571128a505 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:04.462149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1008/2000] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:29:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:29:07.297970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993fce8-e50a-4200-97a6-4b9a21e3fe2b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:07.298462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993fce8-e50a-4200-97a6-4b9a21e3fe2b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:07.298590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993fce8-e50a-4200-97a6-4b9a21e3fe2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:07.317530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4993fce8-e50a-4200-97a6-4b9a21e3fe2b None None] MFA Rules not processed for user `315f9ad503b8402090faac7b827b5463`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:07.318702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4993fce8-e50a-4200-97a6-4b9a21e3fe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:07.325552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 993/2001] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:29:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:29:11.557739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3c88b-f506-4192-8e0b-a665cd0bc531 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:11.558227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3c88b-f506-4192-8e0b-a665cd0bc531 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:11.558324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3c88b-f506-4192-8e0b-a665cd0bc531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:11.578228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-75c3c88b-f506-4192-8e0b-a665cd0bc531 None None] MFA Rules not processed for user `feb837dc77134d96be67f955683b2d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:11.579413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-75c3c88b-f506-4192-8e0b-a665cd0bc531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:11.632424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1009/2002] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:29:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:29:17.312674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=14 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:21.625902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:23.608119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993fce8-e50a-4200-97a6-4b9a21e3fe2b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:23.618370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993fce8-e50a-4200-97a6-4b9a21e3fe2b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:23.621961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:23.622593 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:23.623056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:23.623341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:23.625239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:23.697536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:23.698965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:23.708716 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 994/2003] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:29:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:29:33.704401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=30 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:29:51.178750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37f063-653c-4100-bd55-1a8bac5912c9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.179301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37f063-653c-4100-bd55-1a8bac5912c9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.179301 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37f063-653c-4100-bd55-1a8bac5912c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.200042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ba37f063-653c-4100-bd55-1a8bac5912c9 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:29:51.201556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ba37f063-653c-4100-bd55-1a8bac5912c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:51.273588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1010/2004] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:29:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:29:51.313325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:29:51.314706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:29:51.382276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2f3bb-ce40-40a9-b7f1-54d269b14f6d service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:29:51.387174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f4500-57d6-4301-92d7-ff9d7fc40550 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:29:51.387770 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f4500-57d6-4301-92d7-ff9d7fc40550 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:29:51.388047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f4500-57d6-4301-92d7-ff9d7fc40550 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:29:51.388354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f4500-57d6-4301-92d7-ff9d7fc40550 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:29:51.397735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f4500-57d6-4301-92d7-ff9d7fc40550 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:29:51.399561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f4500-57d6-4301-92d7-ff9d7fc40550 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:29:51.408260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 995/2005] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:29:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:30:00.671526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c8a13-71a3-4b05-ba6c-3489e3786a33 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:00.672137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c8a13-71a3-4b05-ba6c-3489e3786a33 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:00.672137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c8a13-71a3-4b05-ba6c-3489e3786a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:00.692402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3b5c8a13-71a3-4b05-ba6c-3489e3786a33 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:00.693957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3b5c8a13-71a3-4b05-ba6c-3489e3786a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:00.702023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1011/2006] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:30:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:30:00.720251 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f4500-57d6-4301-92d7-ff9d7fc40550 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:00.731546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f4500-57d6-4301-92d7-ff9d7fc40550 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:00.737260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83496-3e73-42d9-8ebb-6acbfba2873a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:00.737859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83496-3e73-42d9-8ebb-6acbfba2873a service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:00.738132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83496-3e73-42d9-8ebb-6acbfba2873a service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:00.738403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba83496-3e73-42d9-8ebb-6acbfba2873a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:00.763164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba83496-3e73-42d9-8ebb-6acbfba2873a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:00.765329 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba83496-3e73-42d9-8ebb-6acbfba2873a service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:00.779537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 996/2007] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:30:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:30:10.338223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325514d-4125-45f1-b0de-5609796a2c7d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.338764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325514d-4125-45f1-b0de-5609796a2c7d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.338895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325514d-4125-45f1-b0de-5609796a2c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.360699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2325514d-4125-45f1-b0de-5609796a2c7d None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:10.362277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2325514d-4125-45f1-b0de-5609796a2c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:10.370070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1012/2008] 10.222.0.48 () {58 vars in 967 bytes} [Mon Oct 24 13:30:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:30:10.388616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83496-3e73-42d9-8ebb-6acbfba2873a service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:10.400667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba83496-3e73-42d9-8ebb-6acbfba2873a service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:10.406441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea1f01-44ac-403e-862d-0cc2e6f77d75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:10.407042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea1f01-44ac-403e-862d-0cc2e6f77d75 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:10.407322 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea1f01-44ac-403e-862d-0cc2e6f77d75 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:10.407596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea1f01-44ac-403e-862d-0cc2e6f77d75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:10.422828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea1f01-44ac-403e-862d-0cc2e6f77d75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:10.425676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea1f01-44ac-403e-862d-0cc2e6f77d75 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:10.434771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 997/2009] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:30:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:30:14.083857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325514d-4125-45f1-b0de-5609796a2c7d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:14.085319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2325514d-4125-45f1-b0de-5609796a2c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:14.097722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=32 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:14.112886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325514d-4125-45f1-b0de-5609796a2c7d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:14.117520 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:14.118137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:14.118624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:14.119058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] PATH_INFO: `/v3/users/feb837dc77134d96be67f955683b2d7d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:14.120172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb837dc77134d96be67f955683b2d7d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:14.122094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:14.363217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb837dc77134d96be67f955683b2d7d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:14.363869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb837dc77134d96be67f955683b2d7d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:14.375979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb837dc77134d96be67f955683b2d7d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:14.383383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb837dc77134d96be67f955683b2d7d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:14.384055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6f647dcf-295f-4043-9b05-283dbe07712b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb837dc77134d96be67f955683b2d7d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:14.469043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1013/2010] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:30:14 2022] DELETE /identity/v3/users/feb837dc77134d96be67f955683b2d7d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:30:14.711071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea1f01-44ac-403e-862d-0cc2e6f77d75 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:14.717057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=45 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:14.723723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea1f01-44ac-403e-862d-0cc2e6f77d75 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:14.800038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:14.805944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:14.805944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:14.805944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] PATH_INFO: `/v3/projects/10316acf2adf4f34a63a08f0e60b2548` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:14.805944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10316acf2adf4f34a63a08f0e60b2548)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:14.805944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:14.934798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10316acf2adf4f34a63a08f0e60b2548', 'request_id': 'req-09528108-528b-4508-93b1-53856d1de258'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:14.941245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10316acf2adf4f34a63a08f0e60b2548', 'request_id': 'req-09528108-528b-4508-93b1-53856d1de258'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:14.947734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10316acf2adf4f34a63a08f0e60b2548', 'request_id': 'req-09528108-528b-4508-93b1-53856d1de258'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:15.033997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] The token cache is being invalidate because project 10316acf2adf4f34a63a08f0e60b2548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:30:15.034507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:15.035316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-09528108-528b-4508-93b1-53856d1de258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:30:15.041319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 998/2011] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:30:14 2022] DELETE /identity/v3/projects/10316acf2adf4f34a63a08f0e60b2548 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:30:15.047042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783435ca-1ae0-4fea-a11a-66fd5c6eec1a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.047042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783435ca-1ae0-4fea-a11a-66fd5c6eec1a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.047042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783435ca-1ae0-4fea-a11a-66fd5c6eec1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.067560 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-783435ca-1ae0-4fea-a11a-66fd5c6eec1a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:15.068744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-783435ca-1ae0-4fea-a11a-66fd5c6eec1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:15.133159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1014/2012] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:30:15.141988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7681e-a48f-4129-a0d1-10af6847147d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.142273 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7681e-a48f-4129-a0d1-10af6847147d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.142546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7681e-a48f-4129-a0d1-10af6847147d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.161989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f2d7681e-a48f-4129-a0d1-10af6847147d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:15.163175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f2d7681e-a48f-4129-a0d1-10af6847147d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:15.171659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 999/2013] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:30:15.175876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783435ca-1ae0-4fea-a11a-66fd5c6eec1a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:15.185425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783435ca-1ae0-4fea-a11a-66fd5c6eec1a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:15.190652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:15.191241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.191516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.191788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.192461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:30:15.192762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:15.194353 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:15.206373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1015/2014] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:30:15.210698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7681e-a48f-4129-a0d1-10af6847147d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:15.220897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7681e-a48f-4129-a0d1-10af6847147d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:15.225579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99958001-f736-426c-8ad5-7c43372576d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:15.226169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958001-f736-426c-8ad5-7c43372576d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.226446 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958001-f736-426c-8ad5-7c43372576d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.226712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99958001-f736-426c-8ad5-7c43372576d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.227415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99958001-f736-426c-8ad5-7c43372576d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:15.229368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99958001-f736-426c-8ad5-7c43372576d7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:15.321205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1000/2015] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:30:15 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:30:15.325564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:15.335386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a4ab8-98a4-4778-8b27-eccee632ad84 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:15.393321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076e33a-07be-4bef-8919-e95ce4c67da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:15.393931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076e33a-07be-4bef-8919-e95ce4c67da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.394214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076e33a-07be-4bef-8919-e95ce4c67da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.394483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076e33a-07be-4bef-8919-e95ce4c67da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.395117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076e33a-07be-4bef-8919-e95ce4c67da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:15.397081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076e33a-07be-4bef-8919-e95ce4c67da4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:15.406771 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:15.469913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1016/2016] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:30:15 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:30:15.473971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99958001-f736-426c-8ad5-7c43372576d7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:15.485069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99958001-f736-426c-8ad5-7c43372576d7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:15.489855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de809694-8001-4c6e-a314-5cc20b12bda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:15.490508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809694-8001-4c6e-a314-5cc20b12bda2 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.490789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809694-8001-4c6e-a314-5cc20b12bda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.491061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de809694-8001-4c6e-a314-5cc20b12bda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.491759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de809694-8001-4c6e-a314-5cc20b12bda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:15.493414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de809694-8001-4c6e-a314-5cc20b12bda2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:15.503808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1001/2017] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:30:15.507939 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076e33a-07be-4bef-8919-e95ce4c67da4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:15.518237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076e33a-07be-4bef-8919-e95ce4c67da4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:15.523081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb5d84-9551-4651-a412-1f67f424c389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:15.523751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb5d84-9551-4651-a412-1f67f424c389 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.524096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb5d84-9551-4651-a412-1f67f424c389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.524760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb5d84-9551-4651-a412-1f67f424c389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.525325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb5d84-9551-4651-a412-1f67f424c389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:15.527572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb5d84-9551-4651-a412-1f67f424c389 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:15.540765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1017/2018] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:30:15.545007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de809694-8001-4c6e-a314-5cc20b12bda2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:15.555537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de809694-8001-4c6e-a314-5cc20b12bda2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:15.560343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:15.560997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.561275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.561549 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] PATH_INFO: `/v3/projects/97beeb44d855467480ebb98eda59be50/users/6d849d9c22d24e209d1e2bf6eca7ac65/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.577402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97beeb44d855467480ebb98eda59be50, user_id=6d849d9c22d24e209d1e2bf6eca7ac65, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:15.580064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:15.666741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1002/2019] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:30:15 2022] PUT /identity/v3/projects/97beeb44d855467480ebb98eda59be50/users/6d849d9c22d24e209d1e2bf6eca7ac65/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:30:15.678084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d785e5-d6a7-4355-872e-522f3f9649f1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.678377 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d785e5-d6a7-4355-872e-522f3f9649f1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.678665 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d785e5-d6a7-4355-872e-522f3f9649f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.708634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c8d785e5-d6a7-4355-872e-522f3f9649f1 None None] MFA Rules not processed for user `6d849d9c22d24e209d1e2bf6eca7ac65`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:15.709892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c8d785e5-d6a7-4355-872e-522f3f9649f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:15.776114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1018/2020] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:30:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:30:15.790060 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:15.791871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:15.829000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:15.856768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aba59-bc3f-4e3c-bb86-616e268ec5c8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:15.860438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:15.861045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:15.861324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:15.861592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:15.870125 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:15.871579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:15.880953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1003/2021] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:30:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:30:17.040398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d785e5-d6a7-4355-872e-522f3f9649f1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:17.041867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c8d785e5-d6a7-4355-872e-522f3f9649f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:17.097856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d785e5-d6a7-4355-872e-522f3f9649f1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:17.101543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c852d56-bb4c-4471-bfec-c53bf41febbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:17.102136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c852d56-bb4c-4471-bfec-c53bf41febbd service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:17.102411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c852d56-bb4c-4471-bfec-c53bf41febbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:17.102676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c852d56-bb4c-4471-bfec-c53bf41febbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:17.104377 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0c852d56-bb4c-4471-bfec-c53bf41febbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:17.175037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c852d56-bb4c-4471-bfec-c53bf41febbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:17.176478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c852d56-bb4c-4471-bfec-c53bf41febbd service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:17.187914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1019/2022] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:30:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:30:18.365673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:18.367057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:18.379317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce27252-b422-4818-9ca6-162cb645855d service glance] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:18.382943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5757a6-dfa9-498c-bb74-294961d1e787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:18.383525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5757a6-dfa9-498c-bb74-294961d1e787 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:18.383804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5757a6-dfa9-498c-bb74-294961d1e787 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:18.384075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5757a6-dfa9-498c-bb74-294961d1e787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:18.385765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2c5757a6-dfa9-498c-bb74-294961d1e787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:18.399383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5757a6-dfa9-498c-bb74-294961d1e787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:18.400836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5757a6-dfa9-498c-bb74-294961d1e787 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:18.411704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1004/2023] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:30:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:30:23.494964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95812b6-55f8-4600-a669-70f107d682ef None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.495704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95812b6-55f8-4600-a669-70f107d682ef None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.495704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95812b6-55f8-4600-a669-70f107d682ef None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.497262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1020/2024] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:23.503221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3e58-8059-4bc6-bc0e-ba76edaa6126 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.505492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3e58-8059-4bc6-bc0e-ba76edaa6126 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.505492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3e58-8059-4bc6-bc0e-ba76edaa6126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.523214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fb7e3e58-8059-4bc6-bc0e-ba76edaa6126 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:23.524578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fb7e3e58-8059-4bc6-bc0e-ba76edaa6126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:23.534114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1005/2025] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:30:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:30:23.545863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95812b6-55f8-4600-a669-70f107d682ef None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:23.557882 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95812b6-55f8-4600-a669-70f107d682ef None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:23.561435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4dd65-b987-4620-81ab-66f1beaae08b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:23.563059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4dd65-b987-4620-81ab-66f1beaae08b service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.563059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4dd65-b987-4620-81ab-66f1beaae08b service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.563059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4dd65-b987-4620-81ab-66f1beaae08b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.574367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4dd65-b987-4620-81ab-66f1beaae08b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:23.576136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4dd65-b987-4620-81ab-66f1beaae08b service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:23.581694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:23.588803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1021/2026] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:30:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:30:23.767035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e2d1f-f427-402b-a8ce-c2cbd038ed71 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.767240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e2d1f-f427-402b-a8ce-c2cbd038ed71 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.767536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e2d1f-f427-402b-a8ce-c2cbd038ed71 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.768936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1006/2027] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:23.773559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166086f4-4f8b-4fb5-844d-64af88dd68e6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.773848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166086f4-4f8b-4fb5-844d-64af88dd68e6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.774119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166086f4-4f8b-4fb5-844d-64af88dd68e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.794223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-166086f4-4f8b-4fb5-844d-64af88dd68e6 None None] MFA Rules not processed for user `65d4ca1062fe496dbe16d6cd1bcac1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:23.795562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-166086f4-4f8b-4fb5-844d-64af88dd68e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:23.802609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1022/2028] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:30:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:30:23.814034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e2d1f-f427-402b-a8ce-c2cbd038ed71 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:23.824774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e2d1f-f427-402b-a8ce-c2cbd038ed71 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:23.829172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36c5c2-fd60-4147-affb-0ff79ce57dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:23.829753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36c5c2-fd60-4147-affb-0ff79ce57dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.830028 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36c5c2-fd60-4147-affb-0ff79ce57dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.830295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36c5c2-fd60-4147-affb-0ff79ce57dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.837145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36c5c2-fd60-4147-affb-0ff79ce57dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:23.838557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36c5c2-fd60-4147-affb-0ff79ce57dd2 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:23.846627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1007/2029] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:30:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:30:23.927548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8df2a-7b25-414d-b245-77dc31d94395 None None] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.927859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8df2a-7b25-414d-b245-77dc31d94395 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.928157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8df2a-7b25-414d-b245-77dc31d94395 None None] PATH_INFO: `/` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.929559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1023/2030] 10.222.0.48 () {54 vars in 921 bytes} [Mon Oct 24 13:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:30:23.934449 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3064a7-d976-42ad-8235-e85e186bf814 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.934730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3064a7-d976-42ad-8235-e85e186bf814 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.935002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3064a7-d976-42ad-8235-e85e186bf814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:23.955062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6e3064a7-d976-42ad-8235-e85e186bf814 None None] MFA Rules not processed for user `5c5b4b17788944f8a0991cb62b2a7205`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:23.956245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6e3064a7-d976-42ad-8235-e85e186bf814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:23.965612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1008/2031] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Oct 24 13:30:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:30:23.977127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8df2a-7b25-414d-b245-77dc31d94395 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:23.988123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8df2a-7b25-414d-b245-77dc31d94395 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:23.991797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dd15a-b401-44d5-83fe-2470ee01694f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:23.992421 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dd15a-b401-44d5-83fe-2470ee01694f service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:23.992701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dd15a-b401-44d5-83fe-2470ee01694f service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:23.992965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dd15a-b401-44d5-83fe-2470ee01694f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:24.003386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dd15a-b401-44d5-83fe-2470ee01694f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:24.004876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dd15a-b401-44d5-83fe-2470ee01694f service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:24.016235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1024/2032] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:30:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:30:29.311947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3064a7-d976-42ad-8235-e85e186bf814 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:29.313511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6e3064a7-d976-42ad-8235-e85e186bf814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:29.330133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3064a7-d976-42ad-8235-e85e186bf814 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:29.336381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:29.336992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:29.337272 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:29.337542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:29.339269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:29.405307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:29.406904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:29.417294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1009/2033] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:30:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:30:34.011027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:38.018892 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba430de3-7e56-48ec-bc99-d9b48667e7cf None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:38.019480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba430de3-7e56-48ec-bc99-d9b48667e7cf None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:38.019480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba430de3-7e56-48ec-bc99-d9b48667e7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:38.039218 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ba430de3-7e56-48ec-bc99-d9b48667e7cf None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:30:38.040514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ba430de3-7e56-48ec-bc99-d9b48667e7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:38.103654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1025/2034] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:30:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:30:38.116694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:38.118771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:38.190864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ae121-2e4c-472e-9acc-bf4b5f47ec49 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:38.195979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1787e5e-3279-4a88-ac6f-235a9c1e46c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:38.196640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1787e5e-3279-4a88-ac6f-235a9c1e46c0 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:38.196906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1787e5e-3279-4a88-ac6f-235a9c1e46c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:38.197176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1787e5e-3279-4a88-ac6f-235a9c1e46c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:38.205854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1787e5e-3279-4a88-ac6f-235a9c1e46c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:38.207538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1787e5e-3279-4a88-ac6f-235a9c1e46c0 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:38.220438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1010/2035] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:30:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:30:48.096578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=25 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:48.212692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:13.938488 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba430de3-7e56-48ec-bc99-d9b48667e7cf None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:13.940087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ba430de3-7e56-48ec-bc99-d9b48667e7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:13.956661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba430de3-7e56-48ec-bc99-d9b48667e7cf None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:13.962682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b8799-32b0-4ef4-897e-aaaf104cf050 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:13.963270 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b8799-32b0-4ef4-897e-aaaf104cf050 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:13.963597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b8799-32b0-4ef4-897e-aaaf104cf050 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:13.963870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b8799-32b0-4ef4-897e-aaaf104cf050 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:13.965610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4d4b8799-32b0-4ef4-897e-aaaf104cf050 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:13.977424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b8799-32b0-4ef4-897e-aaaf104cf050 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:13.979016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b8799-32b0-4ef4-897e-aaaf104cf050 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:13.990023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1026/2036] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:31:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:31:15.199460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1787e5e-3279-4a88-ac6f-235a9c1e46c0 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:15.208862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1787e5e-3279-4a88-ac6f-235a9c1e46c0 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:15.212358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:15.214608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:15.214608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:15.214608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:15.215150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:15.224837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:15.226232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:15.233990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1011/2037] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:31:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:31:16.925716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bbce3-1568-44ed-b32c-bdbf00bc4b12 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:16.926199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bbce3-1568-44ed-b32c-bdbf00bc4b12 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:16.927192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bbce3-1568-44ed-b32c-bdbf00bc4b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:16.947819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-918bbce3-1568-44ed-b32c-bdbf00bc4b12 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:16.949145 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-918bbce3-1568-44ed-b32c-bdbf00bc4b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:16.956756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1027/2038] 10.222.0.48 () {58 vars in 967 bytes} [Mon Oct 24 13:31:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:31:16.981520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:16.991610 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e12cf-fa52-42b2-aebd-735992a77a44 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:16.996719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeaaeb-cf8d-4945-9e37-e8d864e72dc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:16.997421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaaeb-cf8d-4945-9e37-e8d864e72dc9 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:16.998177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaaeb-cf8d-4945-9e37-e8d864e72dc9 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:16.998177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaaeb-cf8d-4945-9e37-e8d864e72dc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:17.010352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeaaeb-cf8d-4945-9e37-e8d864e72dc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:17.010352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeaaeb-cf8d-4945-9e37-e8d864e72dc9 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:17.016903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1012/2039] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:31:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:31:21.779615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bbce3-1568-44ed-b32c-bdbf00bc4b12 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:21.857201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bbce3-1568-44ed-b32c-bdbf00bc4b12 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:21.861762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:21.862429 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:21.862795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:21.862974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] PATH_INFO: `/v3/users/6d849d9c22d24e209d1e2bf6eca7ac65` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:21.865124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d849d9c22d24e209d1e2bf6eca7ac65)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:21.866612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:22.060103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d849d9c22d24e209d1e2bf6eca7ac65'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.060662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d849d9c22d24e209d1e2bf6eca7ac65'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.072603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d849d9c22d24e209d1e2bf6eca7ac65'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.080004 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d849d9c22d24e209d1e2bf6eca7ac65'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.080668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d849d9c22d24e209d1e2bf6eca7ac65'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.142726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1028/2040] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:31:21 2022] DELETE /identity/v3/users/6d849d9c22d24e209d1e2bf6eca7ac65 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:31:22.147223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51013b-0b99-4a87-bcfd-faecd460cce7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:22.147732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51013b-0b99-4a87-bcfd-faecd460cce7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:22.147830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51013b-0b99-4a87-bcfd-faecd460cce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:22.167096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ee51013b-0b99-4a87-bcfd-faecd460cce7 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:22.168300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ee51013b-0b99-4a87-bcfd-faecd460cce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:22.228247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1013/2041] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:31:22.238031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:22.239459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:22.296408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf991763-ee97-4635-a387-e958bb936048 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:22.299933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb6c21-8b81-43b7-9ccb-8988496ae914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:22.300549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6c21-8b81-43b7-9ccb-8988496ae914 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:22.300825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6c21-8b81-43b7-9ccb-8988496ae914 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:22.301094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6c21-8b81-43b7-9ccb-8988496ae914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:22.310011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb6c21-8b81-43b7-9ccb-8988496ae914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:22.311483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb6c21-8b81-43b7-9ccb-8988496ae914 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:22.320679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1029/2042] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:31:22.757723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51013b-0b99-4a87-bcfd-faecd460cce7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:22.757859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ee51013b-0b99-4a87-bcfd-faecd460cce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:22.769992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51013b-0b99-4a87-bcfd-faecd460cce7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:22.775104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:22.775782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:22.776033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:22.776331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] PATH_INFO: `/v3/projects/97beeb44d855467480ebb98eda59be50` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:22.777443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97beeb44d855467480ebb98eda59be50)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:22.779326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:22.882595 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97beeb44d855467480ebb98eda59be50', 'request_id': 'req-5c61f55c-3edf-492e-9783-ddafc296b328'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.888550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97beeb44d855467480ebb98eda59be50', 'request_id': 'req-5c61f55c-3edf-492e-9783-ddafc296b328'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.895088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97beeb44d855467480ebb98eda59be50', 'request_id': 'req-5c61f55c-3edf-492e-9783-ddafc296b328'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.982163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] The token cache is being invalidate because project 97beeb44d855467480ebb98eda59be50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:31:22.982448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.983048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5c61f55c-3edf-492e-9783-ddafc296b328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:22.984592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1014/2043] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:31:22 2022] DELETE /identity/v3/projects/97beeb44d855467480ebb98eda59be50 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:31:22.990446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba2e5c-ffac-4146-a95d-02b86dcfdb7f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:22.991104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba2e5c-ffac-4146-a95d-02b86dcfdb7f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:22.991104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba2e5c-ffac-4146-a95d-02b86dcfdb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:22.998614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:23.012437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-bbba2e5c-ffac-4146-a95d-02b86dcfdb7f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:23.013621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bbba2e5c-ffac-4146-a95d-02b86dcfdb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:23.073209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1030/2044] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:31:23.081000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cfd22-e564-42c9-b4aa-25a896c4b1f8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.081300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cfd22-e564-42c9-b4aa-25a896c4b1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.081572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cfd22-e564-42c9-b4aa-25a896c4b1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.103844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7f5cfd22-e564-42c9-b4aa-25a896c4b1f8 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:23.105071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7f5cfd22-e564-42c9-b4aa-25a896c4b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:23.113033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1015/2045] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:31:23.117770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba2e5c-ffac-4146-a95d-02b86dcfdb7f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:23.127202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba2e5c-ffac-4146-a95d-02b86dcfdb7f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:23.132497 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:23.133087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.133366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.133637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.134284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:31:23.134571 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:23.136309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:23.147676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1031/2046] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:31:23.151877 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cfd22-e564-42c9-b4aa-25a896c4b1f8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:23.161116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cfd22-e564-42c9-b4aa-25a896c4b1f8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:23.165866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64306d37-b882-42fc-b9b9-b88931e51578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:23.166470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64306d37-b882-42fc-b9b9-b88931e51578 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.166726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64306d37-b882-42fc-b9b9-b88931e51578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.166995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64306d37-b882-42fc-b9b9-b88931e51578 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.167618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64306d37-b882-42fc-b9b9-b88931e51578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:23.169512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64306d37-b882-42fc-b9b9-b88931e51578 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:23.281075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1016/2047] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:31:23 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:31:23.285459 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:23.301537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4396bb-4a0e-4132-b784-90b375eb59fd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:23.359358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe4562-fda1-4558-950d-1eee52445610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:23.359877 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4562-fda1-4558-950d-1eee52445610 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.360148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4562-fda1-4558-950d-1eee52445610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.360452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4562-fda1-4558-950d-1eee52445610 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.361084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe4562-fda1-4558-950d-1eee52445610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:23.363053 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe4562-fda1-4558-950d-1eee52445610 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:23.445691 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1032/2048] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:31:23 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:31:23.449761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64306d37-b882-42fc-b9b9-b88931e51578 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:23.460967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64306d37-b882-42fc-b9b9-b88931e51578 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:23.465650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04760e56-d058-4f17-9a88-ac4f658622f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:23.466244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04760e56-d058-4f17-9a88-ac4f658622f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.466522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04760e56-d058-4f17-9a88-ac4f658622f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.466803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04760e56-d058-4f17-9a88-ac4f658622f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.467435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04760e56-d058-4f17-9a88-ac4f658622f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:23.469251 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04760e56-d058-4f17-9a88-ac4f658622f1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:23.479067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1017/2049] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:31:23.483050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe4562-fda1-4558-950d-1eee52445610 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:23.492830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe4562-fda1-4558-950d-1eee52445610 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:23.497321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf98eed-b036-4229-b65f-814bdc72072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:23.497898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98eed-b036-4229-b65f-814bdc72072b admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.498171 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98eed-b036-4229-b65f-814bdc72072b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.498436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf98eed-b036-4229-b65f-814bdc72072b admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.499059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf98eed-b036-4229-b65f-814bdc72072b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:23.500688 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf98eed-b036-4229-b65f-814bdc72072b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:23.512970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1033/2050] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:31:23.517112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04760e56-d058-4f17-9a88-ac4f658622f1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:23.528569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04760e56-d058-4f17-9a88-ac4f658622f1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:23.533908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:23.534560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.534856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.535151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] PATH_INFO: `/v3/projects/4cea469c95d7486eb5ee2632fe0359ea/users/c0b89067dd0644488fd57ccf60b9f800/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.554872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cea469c95d7486eb5ee2632fe0359ea, user_id=c0b89067dd0644488fd57ccf60b9f800, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:23.558064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:23.560539 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:23.686146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1018/2051] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:31:23 2022] PUT /identity/v3/projects/4cea469c95d7486eb5ee2632fe0359ea/users/c0b89067dd0644488fd57ccf60b9f800/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:31:23.696899 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad46948-339a-41cb-b181-1dab7d3eec55 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.697188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad46948-339a-41cb-b181-1dab7d3eec55 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.697490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad46948-339a-41cb-b181-1dab7d3eec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.725393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8ad46948-339a-41cb-b181-1dab7d3eec55 None None] MFA Rules not processed for user `c0b89067dd0644488fd57ccf60b9f800`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:23.726587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8ad46948-339a-41cb-b181-1dab7d3eec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:23.785146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:23.790303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1034/2052] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:31:23.802227 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:23.804474 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:23.859655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff6383-78c2-41af-baf2-0ad2a5784aff admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:23.863232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bf519-77ad-48a7-ab9b-40da7a04167c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:23.863820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bf519-77ad-48a7-ab9b-40da7a04167c service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.864094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bf519-77ad-48a7-ab9b-40da7a04167c service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.864391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bf519-77ad-48a7-ab9b-40da7a04167c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.872959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bf519-77ad-48a7-ab9b-40da7a04167c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:23.874358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bf519-77ad-48a7-ab9b-40da7a04167c service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:23.883361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1019/2053] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:31:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:31:23.908230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad46948-339a-41cb-b181-1dab7d3eec55 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:23.968935 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad46948-339a-41cb-b181-1dab7d3eec55 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:23.973353 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:23.973970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:23.974243 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:23.974510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] PATH_INFO: `/v3/users/c0b89067dd0644488fd57ccf60b9f800` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:23.975615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b89067dd0644488fd57ccf60b9f800)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:23.977550 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:24.191419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b89067dd0644488fd57ccf60b9f800'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:24.192108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b89067dd0644488fd57ccf60b9f800'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:24.204118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b89067dd0644488fd57ccf60b9f800'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:24.211978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b89067dd0644488fd57ccf60b9f800'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:24.212628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b89067dd0644488fd57ccf60b9f800'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:24.308779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1035/2054] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:31:23 2022] DELETE /identity/v3/users/c0b89067dd0644488fd57ccf60b9f800 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:31:24.313080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2051b7-37d2-4cd9-a6bd-ee6aaf458064 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:24.313560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2051b7-37d2-4cd9-a6bd-ee6aaf458064 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:24.313665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2051b7-37d2-4cd9-a6bd-ee6aaf458064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:24.332929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2f2051b7-37d2-4cd9-a6bd-ee6aaf458064 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:24.334012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2f2051b7-37d2-4cd9-a6bd-ee6aaf458064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:24.393991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1020/2055] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:31:24.403436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:24.405047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:24.463070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3abfd0-ff87-4312-ad06-2b5d5e994880 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:24.466748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c705b6f-3a14-49ef-a69e-b839043ef6e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:24.467450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c705b6f-3a14-49ef-a69e-b839043ef6e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:24.467726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c705b6f-3a14-49ef-a69e-b839043ef6e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:24.468023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c705b6f-3a14-49ef-a69e-b839043ef6e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:24.477594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c705b6f-3a14-49ef-a69e-b839043ef6e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:24.479021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c705b6f-3a14-49ef-a69e-b839043ef6e2 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:24.488178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1036/2056] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:31:24.881636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2051b7-37d2-4cd9-a6bd-ee6aaf458064 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:24.883068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2f2051b7-37d2-4cd9-a6bd-ee6aaf458064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:24.895101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2051b7-37d2-4cd9-a6bd-ee6aaf458064 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:24.899713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:24.900379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:24.900658 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:24.900929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] PATH_INFO: `/v3/projects/4cea469c95d7486eb5ee2632fe0359ea` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:24.902095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cea469c95d7486eb5ee2632fe0359ea)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:24.904029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:25.017003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cea469c95d7486eb5ee2632fe0359ea', 'request_id': 'req-0afb2d48-3586-467a-8b71-010555a80c91'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:25.022612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cea469c95d7486eb5ee2632fe0359ea', 'request_id': 'req-0afb2d48-3586-467a-8b71-010555a80c91'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:25.028866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cea469c95d7486eb5ee2632fe0359ea', 'request_id': 'req-0afb2d48-3586-467a-8b71-010555a80c91'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:25.096728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] The token cache is being invalidate because project 4cea469c95d7486eb5ee2632fe0359ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:31:25.097025 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:25.097659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:31:25.099183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1021/2057] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:31:24 2022] DELETE /identity/v3/projects/4cea469c95d7486eb5ee2632fe0359ea => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:31:28.471845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1980081-3fcd-4e38-96d8-6ca9a404ff8f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:28.472344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1980081-3fcd-4e38-96d8-6ca9a404ff8f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:28.472440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1980081-3fcd-4e38-96d8-6ca9a404ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:28.491696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e1980081-3fcd-4e38-96d8-6ca9a404ff8f None None] MFA Rules not processed for user `315f9ad503b8402090faac7b827b5463`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:31:28.492909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e1980081-3fcd-4e38-96d8-6ca9a404ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:28.505050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:28.546897 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1037/2058] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:31:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:31:28.781958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:28.783410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:28.805471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:28.840800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb2d48-3586-467a-8b71-010555a80c91 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:28.844339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:28.849786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:28.849786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:28.849786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:28.849786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:28.856860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:28.858259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:28.866144 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1022/2059] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:31:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:31:29.245069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1980081-3fcd-4e38-96d8-6ca9a404ff8f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:29.246656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e1980081-3fcd-4e38-96d8-6ca9a404ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:29.323757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1980081-3fcd-4e38-96d8-6ca9a404ff8f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:29.328885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:29.329510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:29.329787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:29.330054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:29.331815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:29.343204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:29.344793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:29.354956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1038/2060] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:31:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:31:38.908416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=17 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:39.254749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:31:39.256289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:39.331594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6c276-fa39-45ab-bb2b-914463b2b363 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:31:39.337340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:31:39.337956 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:31:39.338963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:31:39.339241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:31:39.341001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:31:39.350178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=48 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:31:39.354426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:31:39.356005 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:31:39.366952 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1023/2061] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:31:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:31:49.362279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=34 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:06.095365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:06.096774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:06.111510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfc11f-45dd-402d-9848-7dea15b0309c service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:06.116993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:06.117569 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:06.117845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:06.118114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:06.119964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:06.179092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:06.180738 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:06.189735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1039/2062] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:32:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:32:16.188966 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=28 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:24.913094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:24.925193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd7a47-bd48-4fa5-b26c-ae7ae62dfead service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:24.931195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:24.931813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:24.932093 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:24.932393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:24.934121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:25.006627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:25.008367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:25.022401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1024/2063] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:32:35.020969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=28 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:42.570015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:42.571513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:42.585856 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76393ec-4ba4-4631-89a9-d62cab0dad2a service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:42.590796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:42.591413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:42.591668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:42.591942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:42.593702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:42.605590 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:42.607201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:42.618482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1040/2064] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:32:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:32:42.735769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:42.735769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:42.803700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536f5a4-834f-4ebf-80b6-3ab74370cf98 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:42.808594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:42.809168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:42.809445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:42.809717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] PATH_INFO: `/v3/users/315f9ad503b8402090faac7b827b5463` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:42.810828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=315f9ad503b8402090faac7b827b5463)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:42.812774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:43.006509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '315f9ad503b8402090faac7b827b5463'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.013406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '315f9ad503b8402090faac7b827b5463'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.025589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '315f9ad503b8402090faac7b827b5463'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.079594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '315f9ad503b8402090faac7b827b5463'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.080117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1afb3b29-23fe-4ccc-9cc0-0b8c10b1ddf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '315f9ad503b8402090faac7b827b5463'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.086977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1025/2065] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:32:42 2022] DELETE /identity/v3/users/315f9ad503b8402090faac7b827b5463 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:32:43.327077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:43.338190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac138d9-3a5d-4bc8-8593-a8a2adb02d32 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:43.394288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:43.394905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:43.395183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:43.395453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] PATH_INFO: `/v3/projects/ca3097281d4c43458b408414648233a0` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:43.396582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3097281d4c43458b408414648233a0)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:43.398474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:43.507621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3097281d4c43458b408414648233a0', 'request_id': 'req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.588016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3097281d4c43458b408414648233a0', 'request_id': 'req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.594482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3097281d4c43458b408414648233a0', 'request_id': 'req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.605319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] The token cache is being invalidate because project ca3097281d4c43458b408414648233a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:32:43.605603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.606231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-490347c5-b3a2-4e66-9f3f-e6aa52ed1519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:43.607690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1041/2066] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:32:43 2022] DELETE /identity/v3/projects/ca3097281d4c43458b408414648233a0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:32:43.612540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecf31d-11e7-4899-8e4a-c504a31a32d1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:43.612824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecf31d-11e7-4899-8e4a-c504a31a32d1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:43.613096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecf31d-11e7-4899-8e4a-c504a31a32d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:43.632830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2fecf31d-11e7-4899-8e4a-c504a31a32d1 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:43.634018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2fecf31d-11e7-4899-8e4a-c504a31a32d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:43.691532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1026/2067] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:32:43.699224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69539d-4949-4f39-b805-4e57b704c457 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:43.699528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69539d-4949-4f39-b805-4e57b704c457 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:43.699801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69539d-4949-4f39-b805-4e57b704c457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:43.720221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-aa69539d-4949-4f39-b805-4e57b704c457 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:43.721449 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa69539d-4949-4f39-b805-4e57b704c457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:43.729186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1042/2068] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:32:43.733317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecf31d-11e7-4899-8e4a-c504a31a32d1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:43.742900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecf31d-11e7-4899-8e4a-c504a31a32d1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:43.747458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:43.748073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:43.748384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:43.748659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:43.749297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:43.749596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:43.751170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:43.763023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1027/2069] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:32:43.767120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69539d-4949-4f39-b805-4e57b704c457 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:43.776717 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69539d-4949-4f39-b805-4e57b704c457 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:43.781263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4971ca1-0642-4647-85a9-4746a5b183bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:43.781840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4971ca1-0642-4647-85a9-4746a5b183bf admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:43.782118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4971ca1-0642-4647-85a9-4746a5b183bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:43.782414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4971ca1-0642-4647-85a9-4746a5b183bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:43.783031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4971ca1-0642-4647-85a9-4746a5b183bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:43.785020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4971ca1-0642-4647-85a9-4746a5b183bf admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:43.890321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1043/2070] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:32:43 2022] POST /identity/v3/projects => generated 397 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:32:43.891726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:43.902601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a166a1-abbc-412b-bf59-8b91b12d8274 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:43.906330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:43.960201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356bc7-b4c2-4127-84e6-fb5b338a6073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:43.960829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356bc7-b4c2-4127-84e6-fb5b338a6073 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:43.961113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356bc7-b4c2-4127-84e6-fb5b338a6073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:43.961389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc356bc7-b4c2-4127-84e6-fb5b338a6073 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:43.962047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc356bc7-b4c2-4127-84e6-fb5b338a6073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:43.963954 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc356bc7-b4c2-4127-84e6-fb5b338a6073 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:44.057063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1028/2071] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:32:43 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:32:44.061151 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4971ca1-0642-4647-85a9-4746a5b183bf admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:44.072032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4971ca1-0642-4647-85a9-4746a5b183bf admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:44.076941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ead3a-cbe9-4699-a222-b53463e89f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:44.077518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ead3a-cbe9-4699-a222-b53463e89f7c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:44.077792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ead3a-cbe9-4699-a222-b53463e89f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:44.078056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ead3a-cbe9-4699-a222-b53463e89f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:44.078678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ead3a-cbe9-4699-a222-b53463e89f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:44.080292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ead3a-cbe9-4699-a222-b53463e89f7c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:44.090132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1044/2072] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:32:44.094967 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356bc7-b4c2-4127-84e6-fb5b338a6073 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:44.104425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc356bc7-b4c2-4127-84e6-fb5b338a6073 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:44.109121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dec360-c218-4cec-8a34-c41ea71340d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:44.109698 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dec360-c218-4cec-8a34-c41ea71340d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:44.112607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dec360-c218-4cec-8a34-c41ea71340d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:44.112607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dec360-c218-4cec-8a34-c41ea71340d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:44.112607 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dec360-c218-4cec-8a34-c41ea71340d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:44.113011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dec360-c218-4cec-8a34-c41ea71340d2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:44.122840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1029/2073] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:32:44.126906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ead3a-cbe9-4699-a222-b53463e89f7c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:44.137456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ead3a-cbe9-4699-a222-b53463e89f7c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:44.141995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:44.142625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:44.142901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:44.143172 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] PATH_INFO: `/v3/projects/b5e1e79fd57a443a8020979e49ea928b/users/382efd750980481b89b421c126bd8e3c/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:44.157316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e1e79fd57a443a8020979e49ea928b, user_id=382efd750980481b89b421c126bd8e3c, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:44.160039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:44.228075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1045/2074] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:32:44 2022] PUT /identity/v3/projects/b5e1e79fd57a443a8020979e49ea928b/users/382efd750980481b89b421c126bd8e3c/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:32:44.240144 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df75c5-1315-43a6-a0b3-ceb9ace9afef None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:44.240459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df75c5-1315-43a6-a0b3-ceb9ace9afef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:44.240740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df75c5-1315-43a6-a0b3-ceb9ace9afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:44.268831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-34df75c5-1315-43a6-a0b3-ceb9ace9afef None None] MFA Rules not processed for user `382efd750980481b89b421c126bd8e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:44.270021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-34df75c5-1315-43a6-a0b3-ceb9ace9afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:44.332902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1030/2075] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:32:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:32:44.344058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:44.345670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:44.359514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:44.402255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80adeba-67c5-4bbc-90d6-59958a615231 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:44.405851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd18a9-aac9-49a6-acc8-778876e462e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:44.406439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd18a9-aac9-49a6-acc8-778876e462e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:44.406720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd18a9-aac9-49a6-acc8-778876e462e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:44.406986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd18a9-aac9-49a6-acc8-778876e462e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:44.415849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd18a9-aac9-49a6-acc8-778876e462e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:44.417747 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd18a9-aac9-49a6-acc8-778876e462e6 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:44.427237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1046/2076] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:32:44 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:32:54.330079 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:54.398440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df75c5-1315-43a6-a0b3-ceb9ace9afef None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:54.451440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=20 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:54.459423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df75c5-1315-43a6-a0b3-ceb9ace9afef None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:54.465033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:54.465033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:54.465033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:54.465336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] PATH_INFO: `/v3/users/382efd750980481b89b421c126bd8e3c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:54.466510 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382efd750980481b89b421c126bd8e3c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:54.468432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:54.636994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382efd750980481b89b421c126bd8e3c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:54.643827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382efd750980481b89b421c126bd8e3c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:54.655569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382efd750980481b89b421c126bd8e3c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:54.742362 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382efd750980481b89b421c126bd8e3c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:54.743030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382efd750980481b89b421c126bd8e3c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:54.749348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1031/2077] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:32:54 2022] DELETE /identity/v3/users/382efd750980481b89b421c126bd8e3c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:32:54.754450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e464-e2df-40e5-8007-dd9651072dd5 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:54.754734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e464-e2df-40e5-8007-dd9651072dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:54.755004 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e464-e2df-40e5-8007-dd9651072dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:54.774097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-95e2e464-e2df-40e5-8007-dd9651072dd5 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:54.775323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-95e2e464-e2df-40e5-8007-dd9651072dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:54.834836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1047/2078] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:32:54.844594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:54.845974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:54.904941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5b2c5-8f9c-4946-9dcb-06ac441bb505 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:54.908506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53a9de-2449-4c0e-b8c4-4b406975d44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:54.909106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53a9de-2449-4c0e-b8c4-4b406975d44a service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:54.909380 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53a9de-2449-4c0e-b8c4-4b406975d44a service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:54.909647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53a9de-2449-4c0e-b8c4-4b406975d44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:54.919342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53a9de-2449-4c0e-b8c4-4b406975d44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:54.920811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53a9de-2449-4c0e-b8c4-4b406975d44a service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:54.931768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1032/2079] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:32:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:32:55.383761 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2e464-e2df-40e5-8007-dd9651072dd5 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:55.389614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-95e2e464-e2df-40e5-8007-dd9651072dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:55.398683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2e464-e2df-40e5-8007-dd9651072dd5 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:55.403477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:55.404237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:55.404547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:55.404816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] PATH_INFO: `/v3/projects/b5e1e79fd57a443a8020979e49ea928b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:55.406758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5e1e79fd57a443a8020979e49ea928b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:55.408910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:55.513337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5e1e79fd57a443a8020979e49ea928b', 'request_id': 'req-f6c7eaca-7e44-449d-9963-fbb8a14542ed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:55.605583 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5e1e79fd57a443a8020979e49ea928b', 'request_id': 'req-f6c7eaca-7e44-449d-9963-fbb8a14542ed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:55.612419 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5e1e79fd57a443a8020979e49ea928b', 'request_id': 'req-f6c7eaca-7e44-449d-9963-fbb8a14542ed'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:55.623790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] The token cache is being invalidate because project b5e1e79fd57a443a8020979e49ea928b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:32:55.624075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:55.624740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c7eaca-7e44-449d-9963-fbb8a14542ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:32:55.626256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1048/2080] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:32:55 2022] DELETE /identity/v3/projects/b5e1e79fd57a443a8020979e49ea928b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:32:55.631198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2f6fe-d7ef-44cf-b984-436390dac077 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:55.631692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2f6fe-d7ef-44cf-b984-436390dac077 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:55.631877 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2f6fe-d7ef-44cf-b984-436390dac077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:55.652525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-36b2f6fe-d7ef-44cf-b984-436390dac077 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:55.653721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-36b2f6fe-d7ef-44cf-b984-436390dac077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:55.672011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:55.714743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1033/2081] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:32:55.723032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b4480-20e6-490b-9fd2-967466c2ea1c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:55.723318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b4480-20e6-490b-9fd2-967466c2ea1c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:55.723613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b4480-20e6-490b-9fd2-967466c2ea1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:55.745246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8f9b4480-20e6-490b-9fd2-967466c2ea1c None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:55.746439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8f9b4480-20e6-490b-9fd2-967466c2ea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:55.754784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1049/2082] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:32:55.759249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2f6fe-d7ef-44cf-b984-436390dac077 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:55.768987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2f6fe-d7ef-44cf-b984-436390dac077 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:55.774239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:55.774837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:55.775109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:55.775376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:55.776013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:32:55.776337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:55.777898 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:55.789239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1034/2083] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:32:55.793801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b4480-20e6-490b-9fd2-967466c2ea1c None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:55.803065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b4480-20e6-490b-9fd2-967466c2ea1c None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:55.807785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e7852-a609-404a-aa1a-fc758609825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:55.808398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7852-a609-404a-aa1a-fc758609825f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:55.808679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7852-a609-404a-aa1a-fc758609825f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:55.808951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7852-a609-404a-aa1a-fc758609825f admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:55.809612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e7852-a609-404a-aa1a-fc758609825f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:55.811502 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e7852-a609-404a-aa1a-fc758609825f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:55.990253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1050/2084] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:32:55 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:32:55.994240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:56.004691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bc37e-f524-4f0b-8a8a-7fc08faf3c64 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:56.059085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c528d12-708e-4966-8328-c6f78f1481a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:56.059619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c528d12-708e-4966-8328-c6f78f1481a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:56.059895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c528d12-708e-4966-8328-c6f78f1481a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:56.060168 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c528d12-708e-4966-8328-c6f78f1481a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:56.060833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c528d12-708e-4966-8328-c6f78f1481a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:56.062750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c528d12-708e-4966-8328-c6f78f1481a1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:56.157314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1035/2085] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:32:55 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:32:56.158868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e7852-a609-404a-aa1a-fc758609825f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:56.169545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e7852-a609-404a-aa1a-fc758609825f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:56.174240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06570728-c4f8-49f8-ae7e-e3447bcff6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:56.174837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06570728-c4f8-49f8-ae7e-e3447bcff6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:56.175116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06570728-c4f8-49f8-ae7e-e3447bcff6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:56.175916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06570728-c4f8-49f8-ae7e-e3447bcff6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:56.176585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06570728-c4f8-49f8-ae7e-e3447bcff6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:56.178178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06570728-c4f8-49f8-ae7e-e3447bcff6f8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:56.188411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1051/2086] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:32:56.193131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c528d12-708e-4966-8328-c6f78f1481a1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:56.203471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c528d12-708e-4966-8328-c6f78f1481a1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:56.208731 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655a6d8-bdd9-41ef-a8c2-1223cc6d05c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:56.209501 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655a6d8-bdd9-41ef-a8c2-1223cc6d05c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:56.209902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655a6d8-bdd9-41ef-a8c2-1223cc6d05c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:56.210241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655a6d8-bdd9-41ef-a8c2-1223cc6d05c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:56.210961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655a6d8-bdd9-41ef-a8c2-1223cc6d05c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:56.212676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655a6d8-bdd9-41ef-a8c2-1223cc6d05c8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:56.224001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1036/2087] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:32:56.228494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06570728-c4f8-49f8-ae7e-e3447bcff6f8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:56.239248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06570728-c4f8-49f8-ae7e-e3447bcff6f8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:56.243963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:56.244636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:56.244917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:56.245189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] PATH_INFO: `/v3/projects/a946788afc074f079618f1a848ec0f8c/users/33af9a4e3c29426b90dfd5abebde5ed4/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:56.261948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a946788afc074f079618f1a848ec0f8c, user_id=33af9a4e3c29426b90dfd5abebde5ed4, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:56.266844 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:56.330542 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1052/2088] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:32:56 2022] PUT /identity/v3/projects/a946788afc074f079618f1a848ec0f8c/users/33af9a4e3c29426b90dfd5abebde5ed4/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:32:56.335655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc31c-c0ec-40e5-a73f-0b0a025001bf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:56.336006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc31c-c0ec-40e5-a73f-0b0a025001bf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:56.336336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc31c-c0ec-40e5-a73f-0b0a025001bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:56.355944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4fddc31c-c0ec-40e5-a73f-0b0a025001bf None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:32:56.357197 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4fddc31c-c0ec-40e5-a73f-0b0a025001bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:56.414816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1037/2089] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:32:56.424434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:32:56.426681 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:32:56.448362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:32:56.482266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fda335-49d9-4a8b-9b32-e1f2ab8d8535 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:32:56.485820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24e78f-ef96-4064-8956-e2a8fcdbcbe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:32:56.486407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24e78f-ef96-4064-8956-e2a8fcdbcbe7 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:32:56.486682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24e78f-ef96-4064-8956-e2a8fcdbcbe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:32:56.486954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24e78f-ef96-4064-8956-e2a8fcdbcbe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:32:56.496116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24e78f-ef96-4064-8956-e2a8fcdbcbe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:32:56.501294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24e78f-ef96-4064-8956-e2a8fcdbcbe7 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:32:56.507063 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1053/2090] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:03.156365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4243f1-03cd-49d1-8f15-41d6a1c8d3fc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:03.157047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4243f1-03cd-49d1-8f15-41d6a1c8d3fc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:03.157047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4243f1-03cd-49d1-8f15-41d6a1c8d3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:03.192869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fe4243f1-03cd-49d1-8f15-41d6a1c8d3fc None None] MFA Rules not processed for user `33af9a4e3c29426b90dfd5abebde5ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:03.194076 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fe4243f1-03cd-49d1-8f15-41d6a1c8d3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:03.215918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:03.281720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1038/2091] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:03.283294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24e78f-ef96-4064-8956-e2a8fcdbcbe7 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:03.293763 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24e78f-ef96-4064-8956-e2a8fcdbcbe7 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:03.299628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3016af9-0942-4876-9f21-d99650a513b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:03.301225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3016af9-0942-4876-9f21-d99650a513b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:03.305803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3016af9-0942-4876-9f21-d99650a513b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:03.305803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3016af9-0942-4876-9f21-d99650a513b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:03.305803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3016af9-0942-4876-9f21-d99650a513b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:03.305803 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3016af9-0942-4876-9f21-d99650a513b6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:03.451752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1054/2092] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:33:03 2022] POST /identity/v3/projects => generated 371 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:33:03.455555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4243f1-03cd-49d1-8f15-41d6a1c8d3fc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:03.471145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4243f1-03cd-49d1-8f15-41d6a1c8d3fc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:03.528376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1bf1c-6577-4f25-9508-1f02352eaabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:03.529091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1bf1c-6577-4f25-9508-1f02352eaabb admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:03.529371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1bf1c-6577-4f25-9508-1f02352eaabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:03.529646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1bf1c-6577-4f25-9508-1f02352eaabb admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:03.530290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1bf1c-6577-4f25-9508-1f02352eaabb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:03.532192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1bf1c-6577-4f25-9508-1f02352eaabb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:03.617380 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1039/2093] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:33:03 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:33:03.621495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3016af9-0942-4876-9f21-d99650a513b6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:03.632634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3016af9-0942-4876-9f21-d99650a513b6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:03.637221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7550a-3f33-4d5f-8532-853b702826a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:03.637847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7550a-3f33-4d5f-8532-853b702826a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:03.638158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7550a-3f33-4d5f-8532-853b702826a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:03.638404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7550a-3f33-4d5f-8532-853b702826a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:03.639052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7550a-3f33-4d5f-8532-853b702826a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:03.640689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7550a-3f33-4d5f-8532-853b702826a5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:03.655405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1055/2094] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:33:03.656377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1bf1c-6577-4f25-9508-1f02352eaabb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:03.666908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1bf1c-6577-4f25-9508-1f02352eaabb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:03.671677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8496e-8a3e-4b54-bd78-e3465c0f89e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:03.672516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8496e-8a3e-4b54-bd78-e3465c0f89e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:03.677128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8496e-8a3e-4b54-bd78-e3465c0f89e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:03.677128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8496e-8a3e-4b54-bd78-e3465c0f89e4 admin admin] PATH_INFO: `/v3/projects/b565cd9516694f6d80be1675d6b071d0/users/af8e9a0d60b347229e024eedc35cd8e1/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:03.692393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8496e-8a3e-4b54-bd78-e3465c0f89e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b565cd9516694f6d80be1675d6b071d0, user_id=af8e9a0d60b347229e024eedc35cd8e1, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:03.695070 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8496e-8a3e-4b54-bd78-e3465c0f89e4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:03.790243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1040/2095] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:33:03 2022] PUT /identity/v3/projects/b565cd9516694f6d80be1675d6b071d0/users/af8e9a0d60b347229e024eedc35cd8e1/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:33:03.795069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7550a-3f33-4d5f-8532-853b702826a5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:03.805976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7550a-3f33-4d5f-8532-853b702826a5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:03.869570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47d92b-b2c3-49c4-9499-50af6b485552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:03.870201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d92b-b2c3-49c4-9499-50af6b485552 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:03.870477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d92b-b2c3-49c4-9499-50af6b485552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:03.870744 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d92b-b2c3-49c4-9499-50af6b485552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:03.871380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47d92b-b2c3-49c4-9499-50af6b485552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:03.873029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47d92b-b2c3-49c4-9499-50af6b485552 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:03.884772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1056/2096] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:33:03.888968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8496e-8a3e-4b54-bd78-e3465c0f89e4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:03.901105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8496e-8a3e-4b54-bd78-e3465c0f89e4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:03.905642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:03.906286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:03.906560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:03.906870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] PATH_INFO: `/v3/projects/b565cd9516694f6d80be1675d6b071d0/users/af8e9a0d60b347229e024eedc35cd8e1/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:03.909122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b565cd9516694f6d80be1675d6b071d0, user_id=af8e9a0d60b347229e024eedc35cd8e1, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:03.911801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:03.924821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1041/2097] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:33:03 2022] PUT /identity/v3/projects/b565cd9516694f6d80be1675d6b071d0/users/af8e9a0d60b347229e024eedc35cd8e1/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:33:05.208037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3097-590b-4d7b-aa36-8d61674bca2c None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:05.208505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3097-590b-4d7b-aa36-8d61674bca2c None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:05.208624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b3097-590b-4d7b-aa36-8d61674bca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:05.228617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2c3b3097-590b-4d7b-aa36-8d61674bca2c None None] MFA Rules not processed for user `7c096064e6ec443a9e8755bc2e68dbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:05.231436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2c3b3097-590b-4d7b-aa36-8d61674bca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:05.306843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1057/2098] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:05.699409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:05.701562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:05.786151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ec24-b5d9-4c9f-8f28-dd8285b74dbb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:05.790857 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae93c4b-8f79-4416-a35d-c80872a9cf8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:05.791921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93c4b-8f79-4416-a35d-c80872a9cf8e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:05.792290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93c4b-8f79-4416-a35d-c80872a9cf8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:05.792647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae93c4b-8f79-4416-a35d-c80872a9cf8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:05.794612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7ae93c4b-8f79-4416-a35d-c80872a9cf8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:05.810145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae93c4b-8f79-4416-a35d-c80872a9cf8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:05.812059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae93c4b-8f79-4416-a35d-c80872a9cf8e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:05.823042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1042/2099] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:05.956402 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92430-8d0a-4d1e-83be-d7765ba25e33 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:05.956998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92430-8d0a-4d1e-83be-d7765ba25e33 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:05.957271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd92430-8d0a-4d1e-83be-d7765ba25e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:05.992690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8cd92430-8d0a-4d1e-83be-d7765ba25e33 None None] MFA Rules not processed for user `633c6c68e15f4be9a7289d15cac9de43`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:05.993917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8cd92430-8d0a-4d1e-83be-d7765ba25e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:06.049629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:06.103655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1058/2100] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:10.195229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebd425-b8dc-43a9-9f4a-662c9a7b9ec2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:10.195744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebd425-b8dc-43a9-9f4a-662c9a7b9ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:10.195744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebd425-b8dc-43a9-9f4a-662c9a7b9ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:10.226629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:10.234670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-59ebd425-b8dc-43a9-9f4a-662c9a7b9ec2 None None] MFA Rules not processed for user `af8e9a0d60b347229e024eedc35cd8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:10.234670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-59ebd425-b8dc-43a9-9f4a-662c9a7b9ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:10.313493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1043/2101] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:10.329343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd92430-8d0a-4d1e-83be-d7765ba25e33 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:10.332231 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8cd92430-8d0a-4d1e-83be-d7765ba25e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:10.420516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd92430-8d0a-4d1e-83be-d7765ba25e33 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:10.428695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e1356-07f9-4005-8104-a8598714ae1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:10.429335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e1356-07f9-4005-8104-a8598714ae1b service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:10.429667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e1356-07f9-4005-8104-a8598714ae1b service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:10.429907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e1356-07f9-4005-8104-a8598714ae1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:10.503774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e1356-07f9-4005-8104-a8598714ae1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:10.505524 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e1356-07f9-4005-8104-a8598714ae1b service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:10.516697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1059/2102] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:11.936518 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebd425-b8dc-43a9-9f4a-662c9a7b9ec2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:11.938007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-59ebd425-b8dc-43a9-9f4a-662c9a7b9ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:11.961572 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebd425-b8dc-43a9-9f4a-662c9a7b9ec2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:11.968801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:11.969680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:11.970117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:11.970543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:11.973106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:11.993794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:11.995400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:12.006950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1044/2103] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:13.772428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884731c-5eed-47cd-80a2-cc8b1efef596 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:13.772934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884731c-5eed-47cd-80a2-cc8b1efef596 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:13.773044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884731c-5eed-47cd-80a2-cc8b1efef596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:13.793936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-0884731c-5eed-47cd-80a2-cc8b1efef596 None None] MFA Rules not processed for user `33af9a4e3c29426b90dfd5abebde5ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:13.795120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0884731c-5eed-47cd-80a2-cc8b1efef596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:13.817431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1060/2104] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:33:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:33:22.002032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=34 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:23.786591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=70 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:29.023754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:29.025159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:29.161363 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98858db6-7d86-4235-8a45-8b27a2c80173 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:29.166445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:29.167040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:29.167313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:29.167585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:29.169295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:29.180787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:29.182334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:29.192643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1045/2105] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:39.187934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=63 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:43.532930 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b246f-15f1-4588-8f17-770aadacda07 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:43.533478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b246f-15f1-4588-8f17-770aadacda07 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:43.533478 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b246f-15f1-4588-8f17-770aadacda07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:43.552746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-750b246f-15f1-4588-8f17-770aadacda07 None None] MFA Rules not processed for user `e0dd5e32e4a642c8ac9e44310e0a6a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:33:43.553930 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-750b246f-15f1-4588-8f17-770aadacda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:43.609250 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1061/2106] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:33:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:33:45.602217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:33:45.631410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e91424-22e5-4a21-9a9d-e0733c3ab579 service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:33:45.637564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:33:45.638545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:33:45.638856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:33:45.640637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:33:45.642603 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:33:45.717790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:33:45.719365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:33:45.740357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1046/2107] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:33:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:33:53.602672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=25 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:33:55.724692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=35 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:01.555246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51518022-3e2b-4751-80d3-4c90e1fe25ab None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:01.555823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51518022-3e2b-4751-80d3-4c90e1fe25ab None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:01.555823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51518022-3e2b-4751-80d3-4c90e1fe25ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:01.586964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-51518022-3e2b-4751-80d3-4c90e1fe25ab None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:01.588159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-51518022-3e2b-4751-80d3-4c90e1fe25ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:01.644057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1062/2108] 10.222.0.48 () {58 vars in 967 bytes} [Mon Oct 24 13:34:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:01.804037 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:01.805459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:01.875532 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642ebad-912b-4d1b-9964-c2af4a12bc62 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:01.881033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:01.882032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:01.882032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:01.882326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:01.890646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:01.895328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:01.901279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1047/2109] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:34:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:34:02.904279 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51518022-3e2b-4751-80d3-4c90e1fe25ab None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:02.923651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51518022-3e2b-4751-80d3-4c90e1fe25ab None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:02.927403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:02.928006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:02.928403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:02.928679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:02.930602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:03.017792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:03.019165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:03.039491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1063/2110] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:10.856519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:10.858534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:10.873722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3468-d467-41c3-abe2-a1353b52523f service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:10.878159 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19df004-637f-412a-b836-271450785a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:10.878771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19df004-637f-412a-b836-271450785a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:10.879044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19df004-637f-412a-b836-271450785a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:10.879312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19df004-637f-412a-b836-271450785a13 admin admin] PATH_INFO: `/v3/users/834b92eb61864d1b95e9f037379c2e13` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:10.880470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19df004-637f-412a-b836-271450785a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834b92eb61864d1b95e9f037379c2e13)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:10.882353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19df004-637f-412a-b836-271450785a13 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:11.029359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a19df004-637f-412a-b836-271450785a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834b92eb61864d1b95e9f037379c2e13'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.038128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a19df004-637f-412a-b836-271450785a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834b92eb61864d1b95e9f037379c2e13'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.053575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a19df004-637f-412a-b836-271450785a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834b92eb61864d1b95e9f037379c2e13'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.126406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a19df004-637f-412a-b836-271450785a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834b92eb61864d1b95e9f037379c2e13'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.128813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a19df004-637f-412a-b836-271450785a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834b92eb61864d1b95e9f037379c2e13'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.138483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1048/2111] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:34:10 2022] DELETE /identity/v3/users/834b92eb61864d1b95e9f037379c2e13 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:11.412963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:11.424734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806588-b06c-497b-9135-3b299ba0c2d0 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:11.430089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19df004-637f-412a-b836-271450785a13 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:11.431499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a19df004-637f-412a-b836-271450785a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:11.497077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:11.506446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:11.507185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:11.507540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:11.507884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] PATH_INFO: `/v3/projects/1742dccd5a764a31ab1a228ead5cdab9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:11.509112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1742dccd5a764a31ab1a228ead5cdab9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:11.511084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:11.519342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19df004-637f-412a-b836-271450785a13 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:11.523407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:11.523997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:11.524286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:11.524562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:11.526269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:11.531133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:11.591357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:11.592914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:11.602383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1049/2112] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:11.653643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1742dccd5a764a31ab1a228ead5cdab9', 'request_id': 'req-15f60330-6137-4da6-be9f-3eb4c072bd9b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.770592 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1742dccd5a764a31ab1a228ead5cdab9', 'request_id': 'req-15f60330-6137-4da6-be9f-3eb4c072bd9b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.777513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1742dccd5a764a31ab1a228ead5cdab9', 'request_id': 'req-15f60330-6137-4da6-be9f-3eb4c072bd9b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.788706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] The token cache is being invalidate because project 1742dccd5a764a31ab1a228ead5cdab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:34:11.788996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.789637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:11.791127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1064/2113] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:34:11 2022] DELETE /identity/v3/projects/1742dccd5a764a31ab1a228ead5cdab9 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:11.795248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:11.796719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:11.811808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bfb6b-d987-4fef-989f-51a9560d1a55 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:11.870659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:11.871344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:11.871555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:11.872103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] PATH_INFO: `/v3/users/bdea9b919fc549bc9771606ab6e8a5af` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:11.873250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdea9b919fc549bc9771606ab6e8a5af)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:11.875146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:12.047150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdea9b919fc549bc9771606ab6e8a5af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.054985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdea9b919fc549bc9771606ab6e8a5af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.068145 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdea9b919fc549bc9771606ab6e8a5af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.147632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdea9b919fc549bc9771606ab6e8a5af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.148246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d5e6479c-0997-44ad-b100-2a125b0c5249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdea9b919fc549bc9771606ab6e8a5af'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.154593 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1050/2114] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:34:11 2022] DELETE /identity/v3/users/bdea9b919fc549bc9771606ab6e8a5af => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:12.429963 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:12.431468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:12.446198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f60330-6137-4da6-be9f-3eb4c072bd9b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:12.509010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:12.509561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:12.509838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:12.510278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] PATH_INFO: `/v3/projects/bb17d85bda0745ecbc2ba5f99f664d6f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:12.511452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb17d85bda0745ecbc2ba5f99f664d6f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:12.513383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:12.618125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb17d85bda0745ecbc2ba5f99f664d6f', 'request_id': 'req-aee06738-3305-40a5-b78d-5e5e6a39b512'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.684604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb17d85bda0745ecbc2ba5f99f664d6f', 'request_id': 'req-aee06738-3305-40a5-b78d-5e5e6a39b512'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.691593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb17d85bda0745ecbc2ba5f99f664d6f', 'request_id': 'req-aee06738-3305-40a5-b78d-5e5e6a39b512'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.703159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] The token cache is being invalidate because project bb17d85bda0745ecbc2ba5f99f664d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:34:12.703539 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.704189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aee06738-3305-40a5-b78d-5e5e6a39b512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:12.705706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1065/2115] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:34:12 2022] DELETE /identity/v3/projects/bb17d85bda0745ecbc2ba5f99f664d6f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:12.711074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40da2f-213f-4982-b018-e2a660f4c339 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:12.711357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40da2f-213f-4982-b018-e2a660f4c339 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:12.711630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40da2f-213f-4982-b018-e2a660f4c339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:12.742516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:12.774059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2a40da2f-213f-4982-b018-e2a660f4c339 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:12.775690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a40da2f-213f-4982-b018-e2a660f4c339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:12.837640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1051/2116] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:12.846517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862e45-3f10-4898-b5d2-9aa3e02dd360 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:12.846804 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862e45-3f10-4898-b5d2-9aa3e02dd360 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:12.847079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78862e45-3f10-4898-b5d2-9aa3e02dd360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:12.875657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-78862e45-3f10-4898-b5d2-9aa3e02dd360 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:12.876907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-78862e45-3f10-4898-b5d2-9aa3e02dd360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:12.885607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1066/2117] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:12.889667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40da2f-213f-4982-b018-e2a660f4c339 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:12.899774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40da2f-213f-4982-b018-e2a660f4c339 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:12.904420 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:12.905014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:12.905308 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:12.905560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:12.906202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:34:12.906618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:12.908206 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:12.919684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1052/2118] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:34:12.926132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78862e45-3f10-4898-b5d2-9aa3e02dd360 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:12.935412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78862e45-3f10-4898-b5d2-9aa3e02dd360 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:12.940295 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84e0c9-4567-44d3-a9ae-281ffaced7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:12.940980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84e0c9-4567-44d3-a9ae-281ffaced7b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:12.941261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84e0c9-4567-44d3-a9ae-281ffaced7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:12.941532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84e0c9-4567-44d3-a9ae-281ffaced7b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:12.942682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84e0c9-4567-44d3-a9ae-281ffaced7b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:12.944609 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84e0c9-4567-44d3-a9ae-281ffaced7b2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:12.949113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:13.068410 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1067/2119] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:12 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:34:13.072587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:13.097135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb5391-878b-4561-84a7-4f7fcdf30d2c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:13.165609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46657160-4249-4eb8-b0ec-b5c0905f9edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:13.166214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46657160-4249-4eb8-b0ec-b5c0905f9edd admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:13.166495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46657160-4249-4eb8-b0ec-b5c0905f9edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:13.166800 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46657160-4249-4eb8-b0ec-b5c0905f9edd admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:13.167429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46657160-4249-4eb8-b0ec-b5c0905f9edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:13.169374 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46657160-4249-4eb8-b0ec-b5c0905f9edd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:13.284986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1053/2120] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:13 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:13.289812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84e0c9-4567-44d3-a9ae-281ffaced7b2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:13.300163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84e0c9-4567-44d3-a9ae-281ffaced7b2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:13.304786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfa125-7e30-4239-aaa2-314aca0cba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:13.305382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa125-7e30-4239-aaa2-314aca0cba1f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:13.305664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa125-7e30-4239-aaa2-314aca0cba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:13.305974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa125-7e30-4239-aaa2-314aca0cba1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:13.306570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfa125-7e30-4239-aaa2-314aca0cba1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:13.308164 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfa125-7e30-4239-aaa2-314aca0cba1f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:13.318421 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1068/2121] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:13.322241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46657160-4249-4eb8-b0ec-b5c0905f9edd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:13.333461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46657160-4249-4eb8-b0ec-b5c0905f9edd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:13.338391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d010b6c-8de6-4ea1-87de-64baa5f46aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:13.339089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010b6c-8de6-4ea1-87de-64baa5f46aea admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:13.339373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010b6c-8de6-4ea1-87de-64baa5f46aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:13.339648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d010b6c-8de6-4ea1-87de-64baa5f46aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:13.340445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d010b6c-8de6-4ea1-87de-64baa5f46aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:13.342033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d010b6c-8de6-4ea1-87de-64baa5f46aea admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:13.353628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1054/2122] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:13.357770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfa125-7e30-4239-aaa2-314aca0cba1f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:13.367861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfa125-7e30-4239-aaa2-314aca0cba1f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:13.372610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:13.373302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:13.373539 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:13.373812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] PATH_INFO: `/v3/projects/46d41e46b7e3415684c32b22f31eefb5/users/cbdf62fbe49d44a58c294ac2c8b22ef5/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:13.389499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d41e46b7e3415684c32b22f31eefb5, user_id=cbdf62fbe49d44a58c294ac2c8b22ef5, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:13.392021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:13.513479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1069/2123] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:13 2022] PUT /identity/v3/projects/46d41e46b7e3415684c32b22f31eefb5/users/cbdf62fbe49d44a58c294ac2c8b22ef5/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:13.518527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706412b-bbde-4b6a-8816-94c35ecfc212 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:13.521699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706412b-bbde-4b6a-8816-94c35ecfc212 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:13.521699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706412b-bbde-4b6a-8816-94c35ecfc212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:13.539764 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2706412b-bbde-4b6a-8816-94c35ecfc212 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:13.541053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2706412b-bbde-4b6a-8816-94c35ecfc212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:13.603976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:13.619588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1055/2124] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:34:13.628631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:13.630906 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:13.707051 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6b8d7-54be-4d2e-9eb0-4265e79b7b1a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:13.712647 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1358-903c-4b0f-ae0c-e2f43e399dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:13.713249 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1358-903c-4b0f-ae0c-e2f43e399dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:13.713533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1358-903c-4b0f-ae0c-e2f43e399dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:13.713810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e1358-903c-4b0f-ae0c-e2f43e399dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:13.724855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1358-903c-4b0f-ae0c-e2f43e399dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:13.726304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e1358-903c-4b0f-ae0c-e2f43e399dd2 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:13.736863 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1070/2125] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:19.325781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da8bbd-10f7-4ae0-a415-9bcd206e3758 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:19.326480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da8bbd-10f7-4ae0-a415-9bcd206e3758 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:19.326480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da8bbd-10f7-4ae0-a415-9bcd206e3758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:19.363024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-92da8bbd-10f7-4ae0-a415-9bcd206e3758 None None] MFA Rules not processed for user `cbdf62fbe49d44a58c294ac2c8b22ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:19.364317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-92da8bbd-10f7-4ae0-a415-9bcd206e3758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:19.440118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1056/2126] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:19.441113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1358-903c-4b0f-ae0c-e2f43e399dd2 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:19.452934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e1358-903c-4b0f-ae0c-e2f43e399dd2 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:19.457794 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20c4c-b1dd-46fc-a0ca-4ac718e2afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:19.458416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20c4c-b1dd-46fc-a0ca-4ac718e2afa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:19.458691 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20c4c-b1dd-46fc-a0ca-4ac718e2afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:19.458959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20c4c-b1dd-46fc-a0ca-4ac718e2afa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:19.459582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f20c4c-b1dd-46fc-a0ca-4ac718e2afa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:19.461983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f20c4c-b1dd-46fc-a0ca-4ac718e2afa5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:19.589139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1071/2127] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:19 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:19.593337 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da8bbd-10f7-4ae0-a415-9bcd206e3758 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:19.605728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da8bbd-10f7-4ae0-a415-9bcd206e3758 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:19.675580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89e402-62a2-4826-90f2-77af23717bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:19.676184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e402-62a2-4826-90f2-77af23717bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:19.676534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e402-62a2-4826-90f2-77af23717bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:19.677342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e402-62a2-4826-90f2-77af23717bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:19.677993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89e402-62a2-4826-90f2-77af23717bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:19.679880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89e402-62a2-4826-90f2-77af23717bdf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:19.774943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1057/2128] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:19 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:34:19.776254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20c4c-b1dd-46fc-a0ca-4ac718e2afa5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:19.787303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20c4c-b1dd-46fc-a0ca-4ac718e2afa5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:19.792153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787debf8-6103-4da7-8b1e-9b5241ac5535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:19.792782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787debf8-6103-4da7-8b1e-9b5241ac5535 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:19.793061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787debf8-6103-4da7-8b1e-9b5241ac5535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:19.793355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787debf8-6103-4da7-8b1e-9b5241ac5535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:19.794181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787debf8-6103-4da7-8b1e-9b5241ac5535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:19.795769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787debf8-6103-4da7-8b1e-9b5241ac5535 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:19.807038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1072/2129] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:19.811269 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89e402-62a2-4826-90f2-77af23717bdf admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:19.824785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89e402-62a2-4826-90f2-77af23717bdf admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:19.830579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adebbf-7575-4b48-b121-2550e2220714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:19.831285 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adebbf-7575-4b48-b121-2550e2220714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:19.831688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adebbf-7575-4b48-b121-2550e2220714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:19.831961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adebbf-7575-4b48-b121-2550e2220714 admin admin] PATH_INFO: `/v3/projects/0fd9e67249d845a8b4d45bcf1dbb14fc/users/4c42ea650d2d4aa3bc2838c3bb16a18c/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:19.849927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adebbf-7575-4b48-b121-2550e2220714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd9e67249d845a8b4d45bcf1dbb14fc, user_id=4c42ea650d2d4aa3bc2838c3bb16a18c, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:19.852602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adebbf-7575-4b48-b121-2550e2220714 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:19.936447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1058/2130] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:19 2022] PUT /identity/v3/projects/0fd9e67249d845a8b4d45bcf1dbb14fc/users/4c42ea650d2d4aa3bc2838c3bb16a18c/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:34:19.940373 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787debf8-6103-4da7-8b1e-9b5241ac5535 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:19.950979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787debf8-6103-4da7-8b1e-9b5241ac5535 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:20.019320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebccfa4-b044-4f7d-bb21-21aac2c3f813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:20.019320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebccfa4-b044-4f7d-bb21-21aac2c3f813 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:20.019320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebccfa4-b044-4f7d-bb21-21aac2c3f813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:20.019320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebccfa4-b044-4f7d-bb21-21aac2c3f813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:20.019683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebccfa4-b044-4f7d-bb21-21aac2c3f813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:20.019862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebccfa4-b044-4f7d-bb21-21aac2c3f813 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:20.032322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1073/2131] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:20.035267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adebbf-7575-4b48-b121-2550e2220714 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:20.046735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adebbf-7575-4b48-b121-2550e2220714 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:20.051421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:20.052097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:20.052409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:20.052686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] PATH_INFO: `/v3/projects/0fd9e67249d845a8b4d45bcf1dbb14fc/users/4c42ea650d2d4aa3bc2838c3bb16a18c/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:20.054630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd9e67249d845a8b4d45bcf1dbb14fc, user_id=4c42ea650d2d4aa3bc2838c3bb16a18c, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:20.057301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:20.067589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1059/2132] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:20 2022] PUT /identity/v3/projects/0fd9e67249d845a8b4d45bcf1dbb14fc/users/4c42ea650d2d4aa3bc2838c3bb16a18c/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:25.900709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2581-436d-4ee6-934b-fe532f6f412b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:25.901238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2581-436d-4ee6-934b-fe532f6f412b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:25.901238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2581-436d-4ee6-934b-fe532f6f412b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:25.932165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-39db2581-436d-4ee6-934b-fe532f6f412b None None] MFA Rules not processed for user `4c42ea650d2d4aa3bc2838c3bb16a18c`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:25.933598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-39db2581-436d-4ee6-934b-fe532f6f412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:25.938265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:26.015327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1074/2133] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:26.032104 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:26.034040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:26.093050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:26.140198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c1f29-9830-4c7c-b768-cdabd0c99b41 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:26.145766 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:26.146452 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:26.146730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:26.147003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:26.208809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:26.210484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:26.222299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1060/2134] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:31.108648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2581-436d-4ee6-934b-fe532f6f412b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:31.110181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-39db2581-436d-4ee6-934b-fe532f6f412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:31.170765 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2581-436d-4ee6-934b-fe532f6f412b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:31.178661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:31.178661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:31.178661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:31.178661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:31.179068 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:31.250894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:31.252254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:31.262445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1075/2135] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:34.999752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:35.001230 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:35.016774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75713b53-38ea-4b4c-b65f-6a7c4a886abb service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:35.021310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:35.021926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:35.022196 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:35.022463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] PATH_INFO: `/v3/users/633c6c68e15f4be9a7289d15cac9de43` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:35.023551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=633c6c68e15f4be9a7289d15cac9de43)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:35.025490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:35.246736 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '633c6c68e15f4be9a7289d15cac9de43'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.254177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '633c6c68e15f4be9a7289d15cac9de43'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.267876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '633c6c68e15f4be9a7289d15cac9de43'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.336000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '633c6c68e15f4be9a7289d15cac9de43'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.336638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-0ee9c0d0-86ba-4ff6-bc33-05272fd3a655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '633c6c68e15f4be9a7289d15cac9de43'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.343150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1061/2136] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:34:34 2022] DELETE /identity/v3/users/633c6c68e15f4be9a7289d15cac9de43 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:35.554445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:35.564559 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680049c-b468-44bb-89d2-6ed6f4f5635d service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:35.614156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:35.623700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:35.625379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:35.625379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:35.625379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] PATH_INFO: `/v3/projects/986792b802264c748e4614c4d2a17680` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:35.625924 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] RBAC: Authorizing `identity:delete_project(project_id=986792b802264c748e4614c4d2a17680)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:35.627799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:35.755334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986792b802264c748e4614c4d2a17680', 'request_id': 'req-f365697d-e8d8-4a95-8d38-ccdfa883f79d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.827366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986792b802264c748e4614c4d2a17680', 'request_id': 'req-f365697d-e8d8-4a95-8d38-ccdfa883f79d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.834290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986792b802264c748e4614c4d2a17680', 'request_id': 'req-f365697d-e8d8-4a95-8d38-ccdfa883f79d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.845290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] The token cache is being invalidate because project 986792b802264c748e4614c4d2a17680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:34:35.845604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.846237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f365697d-e8d8-4a95-8d38-ccdfa883f79d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:35.847714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1076/2137] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:34:35 2022] DELETE /identity/v3/projects/986792b802264c748e4614c4d2a17680 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:35.852697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c1c0-f5f1-4efa-a0bb-a9afe4e2ab50 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:35.852966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c1c0-f5f1-4efa-a0bb-a9afe4e2ab50 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:35.853238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c1c0-f5f1-4efa-a0bb-a9afe4e2ab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:35.873129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e847c1c0-f5f1-4efa-a0bb-a9afe4e2ab50 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:35.874316 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e847c1c0-f5f1-4efa-a0bb-a9afe4e2ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:35.904691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:35.932680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1062/2138] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:35.940474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac8633-ef02-4d61-be25-1d2c3124d041 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:35.940834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac8633-ef02-4d61-be25-1d2c3124d041 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:35.941023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac8633-ef02-4d61-be25-1d2c3124d041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:35.963748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-97ac8633-ef02-4d61-be25-1d2c3124d041 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:35.965005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97ac8633-ef02-4d61-be25-1d2c3124d041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:35.973409 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1077/2139] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:35.977771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847c1c0-f5f1-4efa-a0bb-a9afe4e2ab50 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:35.987588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847c1c0-f5f1-4efa-a0bb-a9afe4e2ab50 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:35.992972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:35.993806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:35.994082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:35.994350 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:35.994981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:34:35.995279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:35.996895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:36.009026 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1063/2140] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:34:36.013328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac8633-ef02-4d61-be25-1d2c3124d041 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:36.022437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac8633-ef02-4d61-be25-1d2c3124d041 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:36.027032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63500e3e-5ea5-4de2-8acb-3f4d99848cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:36.027642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63500e3e-5ea5-4de2-8acb-3f4d99848cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:36.027919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63500e3e-5ea5-4de2-8acb-3f4d99848cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:36.028189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63500e3e-5ea5-4de2-8acb-3f4d99848cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:36.028872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63500e3e-5ea5-4de2-8acb-3f4d99848cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:36.030814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63500e3e-5ea5-4de2-8acb-3f4d99848cb7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:36.114712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1078/2141] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:36 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:36.118981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:36.129289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee3c98-2ad6-4050-a7c3-5f5ae5bc41c9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:36.195162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd33870-538c-4c8f-89d5-2e82dd89cd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:36.195769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33870-538c-4c8f-89d5-2e82dd89cd7b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:36.196050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33870-538c-4c8f-89d5-2e82dd89cd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:36.196351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33870-538c-4c8f-89d5-2e82dd89cd7b admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:36.197029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd33870-538c-4c8f-89d5-2e82dd89cd7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:36.198944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd33870-538c-4c8f-89d5-2e82dd89cd7b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:36.285434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1064/2142] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:36 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:36.290072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63500e3e-5ea5-4de2-8acb-3f4d99848cb7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:36.300884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63500e3e-5ea5-4de2-8acb-3f4d99848cb7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:36.306705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65feeba-306e-457a-80a4-7304c25c64b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:36.307457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65feeba-306e-457a-80a4-7304c25c64b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:36.307587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65feeba-306e-457a-80a4-7304c25c64b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:36.307858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65feeba-306e-457a-80a4-7304c25c64b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:36.308533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65feeba-306e-457a-80a4-7304c25c64b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:36.311072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65feeba-306e-457a-80a4-7304c25c64b5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:36.320211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1079/2143] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:36.324917 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd33870-538c-4c8f-89d5-2e82dd89cd7b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:36.336234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd33870-538c-4c8f-89d5-2e82dd89cd7b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:36.341463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f011fd-be1e-4fce-b937-635eed7f1297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:36.342114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f011fd-be1e-4fce-b937-635eed7f1297 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:36.342393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f011fd-be1e-4fce-b937-635eed7f1297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:36.342661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f011fd-be1e-4fce-b937-635eed7f1297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:36.343496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f011fd-be1e-4fce-b937-635eed7f1297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:36.345149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f011fd-be1e-4fce-b937-635eed7f1297 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:36.356538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1065/2144] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:36.360679 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65feeba-306e-457a-80a4-7304c25c64b5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:36.371284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65feeba-306e-457a-80a4-7304c25c64b5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:36.376128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:36.376805 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:36.377081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:36.377349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] PATH_INFO: `/v3/projects/bf34f9fd75984e5f98c9dc748f10bb23/users/2c51fca02bc340fc987bb43d9b5e1336/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:36.391917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf34f9fd75984e5f98c9dc748f10bb23, user_id=2c51fca02bc340fc987bb43d9b5e1336, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:36.394612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:36.473323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1080/2145] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:36 2022] PUT /identity/v3/projects/bf34f9fd75984e5f98c9dc748f10bb23/users/2c51fca02bc340fc987bb43d9b5e1336/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:36.483919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2ec2b-ed44-4c62-969f-fa741bee3272 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:36.484204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2ec2b-ed44-4c62-969f-fa741bee3272 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:36.484506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2ec2b-ed44-4c62-969f-fa741bee3272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:36.513455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a2c2ec2b-ed44-4c62-969f-fa741bee3272 None None] MFA Rules not processed for user `2c51fca02bc340fc987bb43d9b5e1336`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:36.514655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a2c2ec2b-ed44-4c62-969f-fa741bee3272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:36.578862 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1066/2146] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:36.591095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:36.592837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:36.677054 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c264934-0994-41d8-8ae1-c2f7d276d680 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:36.681209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bb2f5-1d08-4903-9fd7-83b8919f76e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:36.681806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb2f5-1d08-4903-9fd7-83b8919f76e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:36.682086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb2f5-1d08-4903-9fd7-83b8919f76e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:36.682358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bb2f5-1d08-4903-9fd7-83b8919f76e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:36.690499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bb2f5-1d08-4903-9fd7-83b8919f76e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:36.695047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bb2f5-1d08-4903-9fd7-83b8919f76e6 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:36.697191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:36.701952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1081/2147] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:46.576001 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:48.212933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2ec2b-ed44-4c62-969f-fa741bee3272 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:48.273497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2ec2b-ed44-4c62-969f-fa741bee3272 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:48.278638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:48.279248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:48.279525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:48.279794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] PATH_INFO: `/v3/users/2c51fca02bc340fc987bb43d9b5e1336` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:48.280905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c51fca02bc340fc987bb43d9b5e1336)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:48.282826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:48.445978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c51fca02bc340fc987bb43d9b5e1336'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:48.453925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c51fca02bc340fc987bb43d9b5e1336'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:48.466085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c51fca02bc340fc987bb43d9b5e1336'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:48.518548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c51fca02bc340fc987bb43d9b5e1336'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:48.519158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c51fca02bc340fc987bb43d9b5e1336'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:48.525504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1067/2148] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:34:48 2022] DELETE /identity/v3/users/2c51fca02bc340fc987bb43d9b5e1336 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:48.532399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba9c12-4564-428e-a585-df0af7622ec0 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:48.532901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba9c12-4564-428e-a585-df0af7622ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:48.532999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba9c12-4564-428e-a585-df0af7622ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:48.551399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a0ba9c12-4564-428e-a585-df0af7622ec0 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:48.552654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a0ba9c12-4564-428e-a585-df0af7622ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:48.611772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1082/2149] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:48.625535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:48.627002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:48.684421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104cb8-fc41-4949-a630-674682254cd8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:48.688015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a4bcc-6e3a-4a74-b158-4ee0c90a55f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:48.688641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a4bcc-6e3a-4a74-b158-4ee0c90a55f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:48.688921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a4bcc-6e3a-4a74-b158-4ee0c90a55f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:48.689194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a4bcc-6e3a-4a74-b158-4ee0c90a55f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:48.698188 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a4bcc-6e3a-4a74-b158-4ee0c90a55f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:48.699573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a4bcc-6e3a-4a74-b158-4ee0c90a55f7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:48.709275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1068/2150] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:49.075064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba9c12-4564-428e-a585-df0af7622ec0 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:49.124519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a0ba9c12-4564-428e-a585-df0af7622ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:49.124519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba9c12-4564-428e-a585-df0af7622ec0 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:49.124519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:49.126070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.126070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.126070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] PATH_INFO: `/v3/projects/bf34f9fd75984e5f98c9dc748f10bb23` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.126070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf34f9fd75984e5f98c9dc748f10bb23)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:49.126070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:49.193774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf34f9fd75984e5f98c9dc748f10bb23', 'request_id': 'req-2dca5918-17e9-4dba-8e10-619ca133b8e9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:49.255157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf34f9fd75984e5f98c9dc748f10bb23', 'request_id': 'req-2dca5918-17e9-4dba-8e10-619ca133b8e9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:49.262443 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf34f9fd75984e5f98c9dc748f10bb23', 'request_id': 'req-2dca5918-17e9-4dba-8e10-619ca133b8e9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:49.272988 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] The token cache is being invalidate because project bf34f9fd75984e5f98c9dc748f10bb23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:34:49.273272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:49.273905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2dca5918-17e9-4dba-8e10-619ca133b8e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:49.275365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1083/2151] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:34:49 2022] DELETE /identity/v3/projects/bf34f9fd75984e5f98c9dc748f10bb23 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:49.280373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091e08-6b5f-4b8b-a4e5-000dc9e74219 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.280864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091e08-6b5f-4b8b-a4e5-000dc9e74219 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.280962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091e08-6b5f-4b8b-a4e5-000dc9e74219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.300851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-78091e08-6b5f-4b8b-a4e5-000dc9e74219 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:49.302085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-78091e08-6b5f-4b8b-a4e5-000dc9e74219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:49.320185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:49.361186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1069/2152] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:49.370789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fde30-91fe-4739-a17b-b39d8c497758 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.371072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fde30-91fe-4739-a17b-b39d8c497758 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.371344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fde30-91fe-4739-a17b-b39d8c497758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.391128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-7b3fde30-91fe-4739-a17b-b39d8c497758 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:49.392348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-7b3fde30-91fe-4739-a17b-b39d8c497758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:49.400136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1084/2153] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:49.404444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78091e08-6b5f-4b8b-a4e5-000dc9e74219 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:49.414022 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78091e08-6b5f-4b8b-a4e5-000dc9e74219 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:49.419587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:49.420171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.420492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.420748 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.421386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:34:49.421689 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:49.423261 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:49.435234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1070/2154] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:34:49.439303 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fde30-91fe-4739-a17b-b39d8c497758 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:49.448398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fde30-91fe-4739-a17b-b39d8c497758 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:49.453261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45450bf-972e-4410-83aa-69fc0b9c3f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:49.453898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45450bf-972e-4410-83aa-69fc0b9c3f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.454180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45450bf-972e-4410-83aa-69fc0b9c3f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.454450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45450bf-972e-4410-83aa-69fc0b9c3f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.455066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45450bf-972e-4410-83aa-69fc0b9c3f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:49.456990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45450bf-972e-4410-83aa-69fc0b9c3f55 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:49.524785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1085/2155] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:49 2022] POST /identity/v3/projects => generated 363 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:49.528971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:49.539091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd905897-1ee3-4643-9ece-a6dc50dca403 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:49.594744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7ad-cbfa-46df-ae92-93998791ed7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:49.595346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7ad-cbfa-46df-ae92-93998791ed7c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.595624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7ad-cbfa-46df-ae92-93998791ed7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.595896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7ad-cbfa-46df-ae92-93998791ed7c admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.596560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f7ad-cbfa-46df-ae92-93998791ed7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:49.598456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f7ad-cbfa-46df-ae92-93998791ed7c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:49.678538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1071/2156] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:49 2022] POST /identity/v3/users => generated 282 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:49.683330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45450bf-972e-4410-83aa-69fc0b9c3f55 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:49.694061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45450bf-972e-4410-83aa-69fc0b9c3f55 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:49.698891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c87b8-ca0c-4cc8-ba95-b1d06e94f512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:49.699655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c87b8-ca0c-4cc8-ba95-b1d06e94f512 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.699937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c87b8-ca0c-4cc8-ba95-b1d06e94f512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.700485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c87b8-ca0c-4cc8-ba95-b1d06e94f512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.701140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c87b8-ca0c-4cc8-ba95-b1d06e94f512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:49.702723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c87b8-ca0c-4cc8-ba95-b1d06e94f512 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:49.713159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1086/2157] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:49.717361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7ad-cbfa-46df-ae92-93998791ed7c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:49.727361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7ad-cbfa-46df-ae92-93998791ed7c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:49.733027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a976d4-71e4-41ca-b5b4-9cee984577a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:49.733628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a976d4-71e4-41ca-b5b4-9cee984577a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.733906 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a976d4-71e4-41ca-b5b4-9cee984577a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.734177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a976d4-71e4-41ca-b5b4-9cee984577a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.734819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a976d4-71e4-41ca-b5b4-9cee984577a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:49.736493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a976d4-71e4-41ca-b5b4-9cee984577a1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:49.748149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1072/2158] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:49.752801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c87b8-ca0c-4cc8-ba95-b1d06e94f512 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:49.762574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c87b8-ca0c-4cc8-ba95-b1d06e94f512 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:49.767195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ed391-4754-4d25-a0a3-e1663da38365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:49.767834 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ed391-4754-4d25-a0a3-e1663da38365 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.768112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ed391-4754-4d25-a0a3-e1663da38365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.768417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ed391-4754-4d25-a0a3-e1663da38365 admin admin] PATH_INFO: `/v3/projects/2f3ed484b2d541e294c0f3543b7e4a12/users/2ab4a8fd93e74ea898ead5a9ed694c2e/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.783670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ed391-4754-4d25-a0a3-e1663da38365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3ed484b2d541e294c0f3543b7e4a12, user_id=2ab4a8fd93e74ea898ead5a9ed694c2e, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:49.786352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ed391-4754-4d25-a0a3-e1663da38365 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:49.856042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1087/2159] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:49 2022] PUT /identity/v3/projects/2f3ed484b2d541e294c0f3543b7e4a12/users/2ab4a8fd93e74ea898ead5a9ed694c2e/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:34:49.868204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2297e8b-84cd-4252-abe4-0534cee12652 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:49.868640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2297e8b-84cd-4252-abe4-0534cee12652 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:49.868920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2297e8b-84cd-4252-abe4-0534cee12652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:49.895722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d2297e8b-84cd-4252-abe4-0534cee12652 None None] MFA Rules not processed for user `2ab4a8fd93e74ea898ead5a9ed694c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:49.896942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d2297e8b-84cd-4252-abe4-0534cee12652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:49.960083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1073/2160] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:49.965021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ed391-4754-4d25-a0a3-e1663da38365 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:49.985962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:50.025612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ed391-4754-4d25-a0a3-e1663da38365 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:50.030841 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40218642-f054-4d9e-9388-fa85c97fc8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:50.031505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40218642-f054-4d9e-9388-fa85c97fc8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.032132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40218642-f054-4d9e-9388-fa85c97fc8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.032748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40218642-f054-4d9e-9388-fa85c97fc8f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.034678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40218642-f054-4d9e-9388-fa85c97fc8f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:50.036001 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40218642-f054-4d9e-9388-fa85c97fc8f5 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:50.114225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1088/2161] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:49 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:50.118139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2297e8b-84cd-4252-abe4-0534cee12652 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:50.128977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2297e8b-84cd-4252-abe4-0534cee12652 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:50.141223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:50.186083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8ce10-e29d-41b4-a6b6-c82cfa9ca129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:50.186700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8ce10-e29d-41b4-a6b6-c82cfa9ca129 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.186980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8ce10-e29d-41b4-a6b6-c82cfa9ca129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.187255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8ce10-e29d-41b4-a6b6-c82cfa9ca129 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.187890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8ce10-e29d-41b4-a6b6-c82cfa9ca129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:50.189935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8ce10-e29d-41b4-a6b6-c82cfa9ca129 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:50.249455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1074/2162] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:50 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:34:50.253658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40218642-f054-4d9e-9388-fa85c97fc8f5 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:50.264326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40218642-f054-4d9e-9388-fa85c97fc8f5 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:50.268969 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d8986-ba8e-40e7-996d-76bf4b2578a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:50.269556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d8986-ba8e-40e7-996d-76bf4b2578a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.269901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d8986-ba8e-40e7-996d-76bf4b2578a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.270103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d8986-ba8e-40e7-996d-76bf4b2578a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.270735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d8986-ba8e-40e7-996d-76bf4b2578a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:50.272384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d8986-ba8e-40e7-996d-76bf4b2578a9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:50.283326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1089/2163] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:50.287298 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8ce10-e29d-41b4-a6b6-c82cfa9ca129 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:50.297169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8ce10-e29d-41b4-a6b6-c82cfa9ca129 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:50.301774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e68d-b9d9-4458-b6f7-cca2619b7a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:50.302403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e68d-b9d9-4458-b6f7-cca2619b7a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.304427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e68d-b9d9-4458-b6f7-cca2619b7a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.304427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e68d-b9d9-4458-b6f7-cca2619b7a99 admin admin] PATH_INFO: `/v3/projects/3e61b7f42d8f49679a482fc2f231f012/users/55ed499d40374b1b8452ee8b45c035eb/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.317710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e68d-b9d9-4458-b6f7-cca2619b7a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e61b7f42d8f49679a482fc2f231f012, user_id=55ed499d40374b1b8452ee8b45c035eb, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:50.320384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e68d-b9d9-4458-b6f7-cca2619b7a99 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:50.375289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1075/2164] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:50 2022] PUT /identity/v3/projects/3e61b7f42d8f49679a482fc2f231f012/users/55ed499d40374b1b8452ee8b45c035eb/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:50.379155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d8986-ba8e-40e7-996d-76bf4b2578a9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:50.389793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d8986-ba8e-40e7-996d-76bf4b2578a9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:50.444157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ae271-b042-4213-884b-542d8cc125bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:50.444769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae271-b042-4213-884b-542d8cc125bb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.445050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae271-b042-4213-884b-542d8cc125bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.445322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae271-b042-4213-884b-542d8cc125bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.445946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ae271-b042-4213-884b-542d8cc125bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:50.447531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ae271-b042-4213-884b-542d8cc125bb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:50.457797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1090/2165] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:50.462229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e68d-b9d9-4458-b6f7-cca2619b7a99 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:50.473826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e68d-b9d9-4458-b6f7-cca2619b7a99 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:50.478422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:50.479090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.479362 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.482008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] PATH_INFO: `/v3/projects/3e61b7f42d8f49679a482fc2f231f012/users/55ed499d40374b1b8452ee8b45c035eb/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.482008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e61b7f42d8f49679a482fc2f231f012, user_id=55ed499d40374b1b8452ee8b45c035eb, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:50.484086 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:50.494056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1076/2166] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:50 2022] PUT /identity/v3/projects/3e61b7f42d8f49679a482fc2f231f012/users/55ed499d40374b1b8452ee8b45c035eb/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:50.504346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0326d-1e6d-4212-8f8a-21b468129469 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.510723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0326d-1e6d-4212-8f8a-21b468129469 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.510723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0326d-1e6d-4212-8f8a-21b468129469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.533476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-97d0326d-1e6d-4212-8f8a-21b468129469 None None] MFA Rules not processed for user `55ed499d40374b1b8452ee8b45c035eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:50.534598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-97d0326d-1e6d-4212-8f8a-21b468129469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:50.600438 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1091/2167] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:50.611713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:50.613384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:50.669631 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9c16f-113d-4849-995b-d785c57ca45b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:50.673260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ec83-8993-44bb-bd10-ca7d4422af61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:50.673964 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ec83-8993-44bb-bd10-ca7d4422af61 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:50.674243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ec83-8993-44bb-bd10-ca7d4422af61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:50.674536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ec83-8993-44bb-bd10-ca7d4422af61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:50.683785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ec83-8993-44bb-bd10-ca7d4422af61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:50.685215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ec83-8993-44bb-bd10-ca7d4422af61 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:50.694270 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1077/2168] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:51.586428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0326d-1e6d-4212-8f8a-21b468129469 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:51.603720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:51.648743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0326d-1e6d-4212-8f8a-21b468129469 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:51.653716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:51.654328 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:51.654601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:51.654869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] PATH_INFO: `/v3/users/2ab4a8fd93e74ea898ead5a9ed694c2e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:51.655961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab4a8fd93e74ea898ead5a9ed694c2e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:51.657893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:51.816956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab4a8fd93e74ea898ead5a9ed694c2e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:51.817624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab4a8fd93e74ea898ead5a9ed694c2e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:51.830246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab4a8fd93e74ea898ead5a9ed694c2e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:51.837333 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab4a8fd93e74ea898ead5a9ed694c2e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:51.837902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab4a8fd93e74ea898ead5a9ed694c2e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:51.900334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1092/2169] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:34:51 2022] DELETE /identity/v3/users/2ab4a8fd93e74ea898ead5a9ed694c2e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:51.905418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc40d-4bbd-4fee-ad85-33e8ff8809db None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:51.905897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc40d-4bbd-4fee-ad85-33e8ff8809db None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:51.906013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fc40d-4bbd-4fee-ad85-33e8ff8809db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:51.924880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-597fc40d-4bbd-4fee-ad85-33e8ff8809db None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:51.926046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-597fc40d-4bbd-4fee-ad85-33e8ff8809db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:51.984747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1078/2170] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:51.994317 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:51.995720 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:52.052347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ff4fb-66b5-4068-8fad-feb8ede874eb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:52.055889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beb348-9651-4b14-8dbe-2ee6d12b2ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:52.056512 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beb348-9651-4b14-8dbe-2ee6d12b2ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:52.056788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beb348-9651-4b14-8dbe-2ee6d12b2ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:52.057566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beb348-9651-4b14-8dbe-2ee6d12b2ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:52.066671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0beb348-9651-4b14-8dbe-2ee6d12b2ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:52.068117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0beb348-9651-4b14-8dbe-2ee6d12b2ca6 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:52.078383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1093/2171] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:52.422995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fc40d-4bbd-4fee-ad85-33e8ff8809db None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:52.424498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-597fc40d-4bbd-4fee-ad85-33e8ff8809db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:52.437333 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fc40d-4bbd-4fee-ad85-33e8ff8809db None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:52.441988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:52.442618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:52.442896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:52.443164 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] PATH_INFO: `/v3/projects/2f3ed484b2d541e294c0f3543b7e4a12` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:52.447486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3ed484b2d541e294c0f3543b7e4a12)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:52.447486 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:52.455479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:52.558414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3ed484b2d541e294c0f3543b7e4a12', 'request_id': 'req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.564153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3ed484b2d541e294c0f3543b7e4a12', 'request_id': 'req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.570416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3ed484b2d541e294c0f3543b7e4a12', 'request_id': 'req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.637651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] The token cache is being invalidate because project 2f3ed484b2d541e294c0f3543b7e4a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:34:52.637944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.638556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.640044 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1079/2172] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:34:52 2022] DELETE /identity/v3/projects/2f3ed484b2d541e294c0f3543b7e4a12 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:52.644075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beb348-9651-4b14-8dbe-2ee6d12b2ca6 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:52.653635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beb348-9651-4b14-8dbe-2ee6d12b2ca6 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:52.686993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:52.708961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:52.709574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:52.709851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:52.710153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] PATH_INFO: `/v3/users/55ed499d40374b1b8452ee8b45c035eb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:52.711248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ed499d40374b1b8452ee8b45c035eb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:52.713320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:52.841802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ed499d40374b1b8452ee8b45c035eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.842442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ed499d40374b1b8452ee8b45c035eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.855060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ed499d40374b1b8452ee8b45c035eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.861779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ed499d40374b1b8452ee8b45c035eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.862448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cacff77-ac2c-497b-b250-86017991812f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ed499d40374b1b8452ee8b45c035eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:52.907212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1094/2173] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:34:52 2022] DELETE /identity/v3/users/55ed499d40374b1b8452ee8b45c035eb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:53.280940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:53.282155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:53.297295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34176b-6f6e-491f-a4fa-8f8a694abf1d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:53.365050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:53.365678 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.365953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.366221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] PATH_INFO: `/v3/projects/3e61b7f42d8f49679a482fc2f231f012` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:53.367311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e61b7f42d8f49679a482fc2f231f012)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:53.369254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:53.449910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e61b7f42d8f49679a482fc2f231f012', 'request_id': 'req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:53.455761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e61b7f42d8f49679a482fc2f231f012', 'request_id': 'req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:53.462280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e61b7f42d8f49679a482fc2f231f012', 'request_id': 'req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:53.511799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] The token cache is being invalidate because project 3e61b7f42d8f49679a482fc2f231f012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:34:53.512083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:53.512715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c4eb95c2-3d6b-4afc-ac03-a82996c42e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:53.514246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1080/2174] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:34:53 2022] DELETE /identity/v3/projects/3e61b7f42d8f49679a482fc2f231f012 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:53.519180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c22ac-62de-47f1-979f-5f7b1fac4401 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.519409 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c22ac-62de-47f1-979f-5f7b1fac4401 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.519685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c22ac-62de-47f1-979f-5f7b1fac4401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:53.540015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-941c22ac-62de-47f1-979f-5f7b1fac4401 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:53.541254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-941c22ac-62de-47f1-979f-5f7b1fac4401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:53.599341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1095/2175] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:53.606860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900e73a-8599-4b3e-89f9-36b5e0bed6b3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.607137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900e73a-8599-4b3e-89f9-36b5e0bed6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.607410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900e73a-8599-4b3e-89f9-36b5e0bed6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:53.627013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c900e73a-8599-4b3e-89f9-36b5e0bed6b3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:53.628183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c900e73a-8599-4b3e-89f9-36b5e0bed6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:53.636483 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1081/2176] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:53.640447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c22ac-62de-47f1-979f-5f7b1fac4401 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:53.650047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c22ac-62de-47f1-979f-5f7b1fac4401 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:53.654547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:53.655129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.655404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.655677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:53.656341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:34:53.656650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:53.658260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:53.669505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1096/2177] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:34:53.673565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900e73a-8599-4b3e-89f9-36b5e0bed6b3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:53.683012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900e73a-8599-4b3e-89f9-36b5e0bed6b3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:53.687801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970beeb1-026c-48cb-9063-65452b61f008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:53.688412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970beeb1-026c-48cb-9063-65452b61f008 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.688690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970beeb1-026c-48cb-9063-65452b61f008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.688957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970beeb1-026c-48cb-9063-65452b61f008 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:53.689573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970beeb1-026c-48cb-9063-65452b61f008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:53.691448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970beeb1-026c-48cb-9063-65452b61f008 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:53.770144 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:53.777176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1082/2178] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:53 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:34:53.786576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:53.796576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd1199-a716-4317-964b-0088d85c1371 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:53.838268 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:53.852623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c575fe-f7fe-40de-9386-a93cf559ab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:53.853216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c575fe-f7fe-40de-9386-a93cf559ab45 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.853495 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c575fe-f7fe-40de-9386-a93cf559ab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.853766 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c575fe-f7fe-40de-9386-a93cf559ab45 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:53.854439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c575fe-f7fe-40de-9386-a93cf559ab45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:53.856376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c575fe-f7fe-40de-9386-a93cf559ab45 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:53.964200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1097/2179] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:53 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:53.968407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970beeb1-026c-48cb-9063-65452b61f008 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:53.978445 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970beeb1-026c-48cb-9063-65452b61f008 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:53.983892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86128e90-12a6-4c92-99a7-d6b74ac909ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:53.984540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86128e90-12a6-4c92-99a7-d6b74ac909ac admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:53.984813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86128e90-12a6-4c92-99a7-d6b74ac909ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:53.985098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86128e90-12a6-4c92-99a7-d6b74ac909ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:53.990353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86128e90-12a6-4c92-99a7-d6b74ac909ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:53.990353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86128e90-12a6-4c92-99a7-d6b74ac909ac admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:53.997778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1083/2180] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:54.001802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c575fe-f7fe-40de-9386-a93cf559ab45 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.011348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c575fe-f7fe-40de-9386-a93cf559ab45 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.015937 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08a8dc-060f-401e-b17f-782d396f10a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.016663 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08a8dc-060f-401e-b17f-782d396f10a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.016821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08a8dc-060f-401e-b17f-782d396f10a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.017014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08a8dc-060f-401e-b17f-782d396f10a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.017633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08a8dc-060f-401e-b17f-782d396f10a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.019207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08a8dc-060f-401e-b17f-782d396f10a8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.029677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1098/2181] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:54.033769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86128e90-12a6-4c92-99a7-d6b74ac909ac admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.044235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86128e90-12a6-4c92-99a7-d6b74ac909ac admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.049094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03754d4-748d-4869-8c6b-30cf2f93baf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.049729 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03754d4-748d-4869-8c6b-30cf2f93baf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.050008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03754d4-748d-4869-8c6b-30cf2f93baf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.050281 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03754d4-748d-4869-8c6b-30cf2f93baf1 admin admin] PATH_INFO: `/v3/projects/6fba1209a1294b9db6445668bb83de61/users/16ab891152fe4b8dafa2fc4b50da54e8/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.065681 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03754d4-748d-4869-8c6b-30cf2f93baf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fba1209a1294b9db6445668bb83de61, user_id=16ab891152fe4b8dafa2fc4b50da54e8, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.068319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03754d4-748d-4869-8c6b-30cf2f93baf1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.118939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1084/2182] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:54 2022] PUT /identity/v3/projects/6fba1209a1294b9db6445668bb83de61/users/16ab891152fe4b8dafa2fc4b50da54e8/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:34:54.129964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22e2df-fb6b-4c3a-a3cf-a934aee1f2c7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.130244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22e2df-fb6b-4c3a-a3cf-a934aee1f2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.130510 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22e2df-fb6b-4c3a-a3cf-a934aee1f2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.157199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8d22e2df-fb6b-4c3a-a3cf-a934aee1f2c7 None None] MFA Rules not processed for user `16ab891152fe4b8dafa2fc4b50da54e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:54.158374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8d22e2df-fb6b-4c3a-a3cf-a934aee1f2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:54.220341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1099/2183] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:34:54.224903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03754d4-748d-4869-8c6b-30cf2f93baf1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.293071 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03754d4-748d-4869-8c6b-30cf2f93baf1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.300697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad38df-5db3-4881-bd85-341f33f00197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.301351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad38df-5db3-4881-bd85-341f33f00197 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.301639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad38df-5db3-4881-bd85-341f33f00197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.301889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad38df-5db3-4881-bd85-341f33f00197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.302510 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad38df-5db3-4881-bd85-341f33f00197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.304462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad38df-5db3-4881-bd85-341f33f00197 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.367143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb622-f07a-4601-97f2-26ab630b39ee None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.367430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb622-f07a-4601-97f2-26ab630b39ee None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.367707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98eb622-f07a-4601-97f2-26ab630b39ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.386873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a98eb622-f07a-4601-97f2-26ab630b39ee None None] MFA Rules not processed for user `cbdf62fbe49d44a58c294ac2c8b22ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:54.388056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a98eb622-f07a-4601-97f2-26ab630b39ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:54.412090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1085/2184] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:54 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:34:54.416332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad38df-5db3-4881-bd85-341f33f00197 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.460314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1100/2185] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:54.497808 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad38df-5db3-4881-bd85-341f33f00197 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.503980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e501b7-4413-42a6-bed6-de4028db37df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.504609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e501b7-4413-42a6-bed6-de4028db37df admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.504883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e501b7-4413-42a6-bed6-de4028db37df admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.505154 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e501b7-4413-42a6-bed6-de4028db37df admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.505900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e501b7-4413-42a6-bed6-de4028db37df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.507820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e501b7-4413-42a6-bed6-de4028db37df admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.598973 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1086/2186] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:54 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:54.607165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98eb622-f07a-4601-97f2-26ab630b39ee None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.614820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98eb622-f07a-4601-97f2-26ab630b39ee None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.620309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44108de1-fb77-4cf0-b90b-90d20ef9d233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.620561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44108de1-fb77-4cf0-b90b-90d20ef9d233 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.620659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44108de1-fb77-4cf0-b90b-90d20ef9d233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.620888 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44108de1-fb77-4cf0-b90b-90d20ef9d233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.622094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44108de1-fb77-4cf0-b90b-90d20ef9d233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.623428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44108de1-fb77-4cf0-b90b-90d20ef9d233 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.634383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1101/2187] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:54.639046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e501b7-4413-42a6-bed6-de4028db37df admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.652419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e501b7-4413-42a6-bed6-de4028db37df admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.658221 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b5c14-c2cc-4f52-b3a9-5e3c10cede8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.658949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b5c14-c2cc-4f52-b3a9-5e3c10cede8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.659300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b5c14-c2cc-4f52-b3a9-5e3c10cede8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.659667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b5c14-c2cc-4f52-b3a9-5e3c10cede8c admin admin] PATH_INFO: `/v3/projects/9d3e4865d96d4b059f9df217a88d123f/users/083d19b6890d47ba894f0ba38b573c3a/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.675320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b5c14-c2cc-4f52-b3a9-5e3c10cede8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3e4865d96d4b059f9df217a88d123f, user_id=083d19b6890d47ba894f0ba38b573c3a, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.678027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b5c14-c2cc-4f52-b3a9-5e3c10cede8c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.738774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1087/2188] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:54 2022] PUT /identity/v3/projects/9d3e4865d96d4b059f9df217a88d123f/users/083d19b6890d47ba894f0ba38b573c3a/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:54.740176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44108de1-fb77-4cf0-b90b-90d20ef9d233 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.750481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44108de1-fb77-4cf0-b90b-90d20ef9d233 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.796290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:54.809288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d732aa5-8994-4d62-85bd-ead1b026b73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.812810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d732aa5-8994-4d62-85bd-ead1b026b73c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.812810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d732aa5-8994-4d62-85bd-ead1b026b73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.812810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d732aa5-8994-4d62-85bd-ead1b026b73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.812810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d732aa5-8994-4d62-85bd-ead1b026b73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.813166 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d732aa5-8994-4d62-85bd-ead1b026b73c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.826783 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1102/2189] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:54.827839 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b5c14-c2cc-4f52-b3a9-5e3c10cede8c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.841535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b5c14-c2cc-4f52-b3a9-5e3c10cede8c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:54.844262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:54.844998 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.845346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.845683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] PATH_INFO: `/v3/projects/9d3e4865d96d4b059f9df217a88d123f/users/083d19b6890d47ba894f0ba38b573c3a/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.847660 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3e4865d96d4b059f9df217a88d123f, user_id=083d19b6890d47ba894f0ba38b573c3a, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:54.850472 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:54.857216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:54.862193 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1088/2190] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:54 2022] PUT /identity/v3/projects/9d3e4865d96d4b059f9df217a88d123f/users/083d19b6890d47ba894f0ba38b573c3a/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:54.873118 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4cfc8-a7bf-4ee3-b524-49a60db71e5d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:54.873406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4cfc8-a7bf-4ee3-b524-49a60db71e5d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:54.873683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4cfc8-a7bf-4ee3-b524-49a60db71e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:54.902732 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ada4cfc8-a7bf-4ee3-b524-49a60db71e5d None None] MFA Rules not processed for user `083d19b6890d47ba894f0ba38b573c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:54.904016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ada4cfc8-a7bf-4ee3-b524-49a60db71e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:54.975021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1103/2191] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:54.985766 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:54.987556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:55.047279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e55188-741c-404a-b06e-700c6b5c4f9b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:55.050925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65820f29-80a3-4e51-aa49-7219240bd043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:55.051526 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65820f29-80a3-4e51-aa49-7219240bd043 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:55.051803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65820f29-80a3-4e51-aa49-7219240bd043 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:55.052073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65820f29-80a3-4e51-aa49-7219240bd043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:55.061926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65820f29-80a3-4e51-aa49-7219240bd043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:55.063592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65820f29-80a3-4e51-aa49-7219240bd043 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:55.074191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1089/2192] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:55.106082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4cfc8-a7bf-4ee3-b524-49a60db71e5d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:55.169752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4cfc8-a7bf-4ee3-b524-49a60db71e5d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:55.174409 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:55.175033 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:55.175312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:55.175574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] PATH_INFO: `/v3/users/16ab891152fe4b8dafa2fc4b50da54e8` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:55.176775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ab891152fe4b8dafa2fc4b50da54e8)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:55.178742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:55.351915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ab891152fe4b8dafa2fc4b50da54e8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:55.352572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ab891152fe4b8dafa2fc4b50da54e8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:55.365392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ab891152fe4b8dafa2fc4b50da54e8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:55.369959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:55.372980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ab891152fe4b8dafa2fc4b50da54e8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:55.373643 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ab891152fe4b8dafa2fc4b50da54e8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:55.441221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1104/2193] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:34:55 2022] DELETE /identity/v3/users/16ab891152fe4b8dafa2fc4b50da54e8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:55.445781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652ca14-2746-4dd2-bfa8-7de98e5d846b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:55.446287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652ca14-2746-4dd2-bfa8-7de98e5d846b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:55.446453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652ca14-2746-4dd2-bfa8-7de98e5d846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:55.468699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9652ca14-2746-4dd2-bfa8-7de98e5d846b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:55.469953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9652ca14-2746-4dd2-bfa8-7de98e5d846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:55.536672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1090/2194] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:34:55.547212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:55.548726 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:55.631133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbde6c-c14b-4974-8358-6781a7a9698b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:55.635178 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7678d-7003-4ad0-86a1-3e204bcee0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:55.636411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7678d-7003-4ad0-86a1-3e204bcee0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:55.636785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7678d-7003-4ad0-86a1-3e204bcee0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:55.637123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7678d-7003-4ad0-86a1-3e204bcee0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:55.647702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7678d-7003-4ad0-86a1-3e204bcee0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:55.649294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7678d-7003-4ad0-86a1-3e204bcee0d3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:55.659724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1105/2195] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:56.068639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652ca14-2746-4dd2-bfa8-7de98e5d846b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:56.070068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9652ca14-2746-4dd2-bfa8-7de98e5d846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:56.099095 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652ca14-2746-4dd2-bfa8-7de98e5d846b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:56.104245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:56.104950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:56.105387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:56.105709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] PATH_INFO: `/v3/projects/6fba1209a1294b9db6445668bb83de61` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:56.106928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fba1209a1294b9db6445668bb83de61)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:56.109273 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:56.210118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fba1209a1294b9db6445668bb83de61', 'request_id': 'req-acccc66e-7eb3-4159-b1a0-0962aa4e7906'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.215727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fba1209a1294b9db6445668bb83de61', 'request_id': 'req-acccc66e-7eb3-4159-b1a0-0962aa4e7906'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.222143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fba1209a1294b9db6445668bb83de61', 'request_id': 'req-acccc66e-7eb3-4159-b1a0-0962aa4e7906'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.266403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] The token cache is being invalidate because project 6fba1209a1294b9db6445668bb83de61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:34:56.266695 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.267327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.268905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1091/2196] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:34:56 2022] DELETE /identity/v3/projects/6fba1209a1294b9db6445668bb83de61 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:56.272962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7678d-7003-4ad0-86a1-3e204bcee0d3 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:56.283159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7678d-7003-4ad0-86a1-3e204bcee0d3 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:56.339513 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:56.340128 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:56.340434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:56.340703 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] PATH_INFO: `/v3/users/083d19b6890d47ba894f0ba38b573c3a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:56.341820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083d19b6890d47ba894f0ba38b573c3a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:56.343718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:56.520640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083d19b6890d47ba894f0ba38b573c3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.521292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083d19b6890d47ba894f0ba38b573c3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.534862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083d19b6890d47ba894f0ba38b573c3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.543210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083d19b6890d47ba894f0ba38b573c3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.544047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61edee56-8921-4eda-9574-089ea597ede0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083d19b6890d47ba894f0ba38b573c3a'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:56.627773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1106/2197] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:34:56 2022] DELETE /identity/v3/users/083d19b6890d47ba894f0ba38b573c3a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:57.086223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:57.090986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:57.118264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccc66e-7eb3-4159-b1a0-0962aa4e7906 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:57.142254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:57.183923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:57.183923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.183923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.183923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] PATH_INFO: `/v3/projects/9d3e4865d96d4b059f9df217a88d123f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.184444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3e4865d96d4b059f9df217a88d123f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:57.184444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:57.280535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3e4865d96d4b059f9df217a88d123f', 'request_id': 'req-927cce60-aace-4509-954f-66db6b27bb31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:57.285926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3e4865d96d4b059f9df217a88d123f', 'request_id': 'req-927cce60-aace-4509-954f-66db6b27bb31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:57.292733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3e4865d96d4b059f9df217a88d123f', 'request_id': 'req-927cce60-aace-4509-954f-66db6b27bb31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:57.368346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] The token cache is being invalidate because project 9d3e4865d96d4b059f9df217a88d123f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:34:57.368634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:57.369236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-927cce60-aace-4509-954f-66db6b27bb31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:34:57.370756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1092/2198] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:34:57 2022] DELETE /identity/v3/projects/9d3e4865d96d4b059f9df217a88d123f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:57.375680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e2e9c-f54c-4db6-b4e6-2d57672ed69f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.376138 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e2e9c-f54c-4db6-b4e6-2d57672ed69f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.376250 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e2e9c-f54c-4db6-b4e6-2d57672ed69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.395828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-447e2e9c-f54c-4db6-b4e6-2d57672ed69f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:57.397108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-447e2e9c-f54c-4db6-b4e6-2d57672ed69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:57.401941 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:57.458006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1107/2199] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:57.470536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2fcf5-4339-471b-b984-d01077597c59 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.470817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2fcf5-4339-471b-b984-d01077597c59 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.471091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2fcf5-4339-471b-b984-d01077597c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.491171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-88c2fcf5-4339-471b-b984-d01077597c59 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:57.492383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-88c2fcf5-4339-471b-b984-d01077597c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:57.500212 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1093/2200] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:57.504391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e2e9c-f54c-4db6-b4e6-2d57672ed69f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:57.513764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e2e9c-f54c-4db6-b4e6-2d57672ed69f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:57.518223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:57.518790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.520527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.520527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.520527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:34:57.520527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:57.521829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:57.533403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1108/2201] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:34:57.537860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2fcf5-4339-471b-b984-d01077597c59 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:57.548735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2fcf5-4339-471b-b984-d01077597c59 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:57.553487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0db95-264f-465d-ab0a-0e20a886adab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:57.554078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0db95-264f-465d-ab0a-0e20a886adab admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.554365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0db95-264f-465d-ab0a-0e20a886adab admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.554638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0db95-264f-465d-ab0a-0e20a886adab admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.555957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0db95-264f-465d-ab0a-0e20a886adab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:57.558348 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0db95-264f-465d-ab0a-0e20a886adab admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:57.669415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1094/2202] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:57 2022] POST /identity/v3/projects => generated 391 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:34:57.673670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:57.687174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36da6-c5d2-49f8-b85f-2e3c2f5e3fe6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:57.747983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c997482-805b-4210-9f76-e6809001e2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:57.748753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c997482-805b-4210-9f76-e6809001e2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.748889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c997482-805b-4210-9f76-e6809001e2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.749158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c997482-805b-4210-9f76-e6809001e2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.751401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c997482-805b-4210-9f76-e6809001e2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:57.752230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c997482-805b-4210-9f76-e6809001e2a6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:57.848905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1109/2203] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:57 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:57.850535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0db95-264f-465d-ab0a-0e20a886adab admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:57.861536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0db95-264f-465d-ab0a-0e20a886adab admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:57.866241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55b36-1797-4f9b-8fb3-e8dc66d728ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:57.866905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55b36-1797-4f9b-8fb3-e8dc66d728ac admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.867244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55b36-1797-4f9b-8fb3-e8dc66d728ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.867580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f55b36-1797-4f9b-8fb3-e8dc66d728ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.868365 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f55b36-1797-4f9b-8fb3-e8dc66d728ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:57.870003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f55b36-1797-4f9b-8fb3-e8dc66d728ac admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:57.882279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1095/2204] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:57.891334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c997482-805b-4210-9f76-e6809001e2a6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:57.903579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c997482-805b-4210-9f76-e6809001e2a6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:57.908737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da34ab-a4ed-432e-9a80-4f093548d97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:57.909337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da34ab-a4ed-432e-9a80-4f093548d97c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.909624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da34ab-a4ed-432e-9a80-4f093548d97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.909901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da34ab-a4ed-432e-9a80-4f093548d97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.910595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da34ab-a4ed-432e-9a80-4f093548d97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:57.912192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da34ab-a4ed-432e-9a80-4f093548d97c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:57.924772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1110/2205] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:57.929085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55b36-1797-4f9b-8fb3-e8dc66d728ac admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:57.939467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f55b36-1797-4f9b-8fb3-e8dc66d728ac admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:57.944244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36862e6-9a23-457b-8313-bb9d049c2953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:57.945018 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36862e6-9a23-457b-8313-bb9d049c2953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:57.945410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36862e6-9a23-457b-8313-bb9d049c2953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:57.945622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36862e6-9a23-457b-8313-bb9d049c2953 admin admin] PATH_INFO: `/v3/projects/53aebc4cb2b248369514a26a8d2d4ea6/users/cb2d7a5bd8374fe0a8715cbfc8b10347/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:57.961385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36862e6-9a23-457b-8313-bb9d049c2953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53aebc4cb2b248369514a26a8d2d4ea6, user_id=cb2d7a5bd8374fe0a8715cbfc8b10347, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:57.964637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36862e6-9a23-457b-8313-bb9d049c2953 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:58.045416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1096/2206] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:57 2022] PUT /identity/v3/projects/53aebc4cb2b248369514a26a8d2d4ea6/users/cb2d7a5bd8374fe0a8715cbfc8b10347/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:34:58.056318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9f835-23d1-4a12-b8c0-2e494eeb9c10 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.057124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9f835-23d1-4a12-b8c0-2e494eeb9c10 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.057936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9f835-23d1-4a12-b8c0-2e494eeb9c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:58.101312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-e5e9f835-23d1-4a12-b8c0-2e494eeb9c10 None None] MFA Rules not processed for user `cb2d7a5bd8374fe0a8715cbfc8b10347`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:58.102531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e5e9f835-23d1-4a12-b8c0-2e494eeb9c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:58.173425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1111/2207] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:58.180007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36862e6-9a23-457b-8313-bb9d049c2953 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:58.234481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:58.244655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36862e6-9a23-457b-8313-bb9d049c2953 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:58.249936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57b40-2acc-4710-baca-0b2f74b926f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:58.250533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57b40-2acc-4710-baca-0b2f74b926f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.250813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57b40-2acc-4710-baca-0b2f74b926f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.251086 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf57b40-2acc-4710-baca-0b2f74b926f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:58.251733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57b40-2acc-4710-baca-0b2f74b926f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:58.253696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf57b40-2acc-4710-baca-0b2f74b926f7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:58.354288 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1097/2208] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:34:58 2022] POST /identity/v3/projects => generated 393 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:34:58.358437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9f835-23d1-4a12-b8c0-2e494eeb9c10 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:58.365537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:58.369329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9f835-23d1-4a12-b8c0-2e494eeb9c10 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:58.426414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f592-e4ff-45cd-8166-0018dd3b5912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:58.427029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f592-e4ff-45cd-8166-0018dd3b5912 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.427822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f592-e4ff-45cd-8166-0018dd3b5912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.427822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009f592-e4ff-45cd-8166-0018dd3b5912 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:58.428165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009f592-e4ff-45cd-8166-0018dd3b5912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:58.430112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009f592-e4ff-45cd-8166-0018dd3b5912 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:58.514407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1112/2209] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:34:58 2022] POST /identity/v3/users => generated 303 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:34:58.518563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57b40-2acc-4710-baca-0b2f74b926f7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:58.531492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf57b40-2acc-4710-baca-0b2f74b926f7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:58.536010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54b533-c3d4-4eae-b8ba-40649417d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:58.536647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54b533-c3d4-4eae-b8ba-40649417d48d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.536919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54b533-c3d4-4eae-b8ba-40649417d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.537185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54b533-c3d4-4eae-b8ba-40649417d48d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:58.537820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54b533-c3d4-4eae-b8ba-40649417d48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:58.539470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54b533-c3d4-4eae-b8ba-40649417d48d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:58.550573 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1098/2210] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:58.555138 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f592-e4ff-45cd-8166-0018dd3b5912 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:58.566727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009f592-e4ff-45cd-8166-0018dd3b5912 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:58.571814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4464c24-2cc1-4b16-8ee5-20f79b8ccd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:58.572509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4464c24-2cc1-4b16-8ee5-20f79b8ccd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.572853 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4464c24-2cc1-4b16-8ee5-20f79b8ccd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.573162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4464c24-2cc1-4b16-8ee5-20f79b8ccd0e admin admin] PATH_INFO: `/v3/projects/65adfb8b2a9f42b7b03785aa4c1da79b/users/aac21d9d9b5248b7904259ff98820cbd/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:58.589439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4464c24-2cc1-4b16-8ee5-20f79b8ccd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65adfb8b2a9f42b7b03785aa4c1da79b, user_id=aac21d9d9b5248b7904259ff98820cbd, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:58.592214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4464c24-2cc1-4b16-8ee5-20f79b8ccd0e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:58.652318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1113/2211] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:58 2022] PUT /identity/v3/projects/65adfb8b2a9f42b7b03785aa4c1da79b/users/aac21d9d9b5248b7904259ff98820cbd/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:58.656152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54b533-c3d4-4eae-b8ba-40649417d48d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:58.668417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54b533-c3d4-4eae-b8ba-40649417d48d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:58.727078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2bb5c-43e1-4a11-803a-3c05f1b9ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:58.727691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2bb5c-43e1-4a11-803a-3c05f1b9ddf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.727969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2bb5c-43e1-4a11-803a-3c05f1b9ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.728315 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2bb5c-43e1-4a11-803a-3c05f1b9ddf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:58.729012 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2bb5c-43e1-4a11-803a-3c05f1b9ddf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:58.731517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2bb5c-43e1-4a11-803a-3c05f1b9ddf5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:58.742165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1099/2212] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:34:58.746623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4464c24-2cc1-4b16-8ee5-20f79b8ccd0e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:58.758016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4464c24-2cc1-4b16-8ee5-20f79b8ccd0e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:58.763073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:58.763725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.764156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.764492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] PATH_INFO: `/v3/projects/65adfb8b2a9f42b7b03785aa4c1da79b/users/aac21d9d9b5248b7904259ff98820cbd/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:58.766431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=65adfb8b2a9f42b7b03785aa4c1da79b, user_id=aac21d9d9b5248b7904259ff98820cbd, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:58.769535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:58.781698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1114/2213] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:34:58 2022] PUT /identity/v3/projects/65adfb8b2a9f42b7b03785aa4c1da79b/users/aac21d9d9b5248b7904259ff98820cbd/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:34:58.793873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5800-1560-4ba8-a02d-62405a173e14 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.794150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5800-1560-4ba8-a02d-62405a173e14 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.794643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5800-1560-4ba8-a02d-62405a173e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:58.828146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cd8f5800-1560-4ba8-a02d-62405a173e14 None None] MFA Rules not processed for user `aac21d9d9b5248b7904259ff98820cbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:34:58.829624 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cd8f5800-1560-4ba8-a02d-62405a173e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:58.908461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1100/2214] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:34:58 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:34:58.916417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:58.918785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:58.980923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b0b9-99b1-452c-b874-31edb240ba8c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:58.985975 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264a989-0bd1-4ac1-8229-2f36f725f586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:58.986714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264a989-0bd1-4ac1-8229-2f36f725f586 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:58.987116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264a989-0bd1-4ac1-8229-2f36f725f586 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:58.987441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264a989-0bd1-4ac1-8229-2f36f725f586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:59.003395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264a989-0bd1-4ac1-8229-2f36f725f586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:59.004973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264a989-0bd1-4ac1-8229-2f36f725f586 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:59.015380 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1115/2215] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:58 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:34:59.062401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5800-1560-4ba8-a02d-62405a173e14 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:34:59.063796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cd8f5800-1560-4ba8-a02d-62405a173e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:34:59.075882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5800-1560-4ba8-a02d-62405a173e14 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:34:59.079467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcd093-3deb-4e6c-9a8c-26050a1571b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:34:59.080061 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcd093-3deb-4e6c-9a8c-26050a1571b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:34:59.080372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcd093-3deb-4e6c-9a8c-26050a1571b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:34:59.080645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcd093-3deb-4e6c-9a8c-26050a1571b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:34:59.129257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:34:59.142456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcd093-3deb-4e6c-9a8c-26050a1571b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:34:59.144043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcd093-3deb-4e6c-9a8c-26050a1571b7 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:34:59.154326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1101/2216] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:34:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:05.013156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264a989-0bd1-4ac1-8229-2f36f725f586 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:05.072868 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264a989-0bd1-4ac1-8229-2f36f725f586 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:05.078015 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:05.078667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:05.078943 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:05.079211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] PATH_INFO: `/v3/users/cb2d7a5bd8374fe0a8715cbfc8b10347` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:05.080356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2d7a5bd8374fe0a8715cbfc8b10347)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:05.082454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:05.084122 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:05.246187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2d7a5bd8374fe0a8715cbfc8b10347'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:05.246831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2d7a5bd8374fe0a8715cbfc8b10347'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:05.259165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2d7a5bd8374fe0a8715cbfc8b10347'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:05.266954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2d7a5bd8374fe0a8715cbfc8b10347'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:05.267697 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2d7a5bd8374fe0a8715cbfc8b10347'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:05.347056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1116/2217] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:35:05 2022] DELETE /identity/v3/users/cb2d7a5bd8374fe0a8715cbfc8b10347 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:05.351433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e464-0f34-4473-9860-a3a68ddd3e3d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:05.351915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e464-0f34-4473-9860-a3a68ddd3e3d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:05.352013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e464-0f34-4473-9860-a3a68ddd3e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:05.370467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3884e464-0f34-4473-9860-a3a68ddd3e3d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:05.371651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3884e464-0f34-4473-9860-a3a68ddd3e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:05.430628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1102/2218] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:35:05.440540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:05.441948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:05.499213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f94d2-b9ed-46ae-9343-9c052db06b3f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:05.502802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbdc12-2f6f-47d8-a6b0-f57b535eb6b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:05.503391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbdc12-2f6f-47d8-a6b0-f57b535eb6b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:05.503668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbdc12-2f6f-47d8-a6b0-f57b535eb6b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:05.503935 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbdc12-2f6f-47d8-a6b0-f57b535eb6b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:05.513432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbdc12-2f6f-47d8-a6b0-f57b535eb6b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:05.514849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbdc12-2f6f-47d8-a6b0-f57b535eb6b9 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:05.523986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1117/2219] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:05.914171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e464-0f34-4473-9860-a3a68ddd3e3d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:05.915617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3884e464-0f34-4473-9860-a3a68ddd3e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:05.929245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e464-0f34-4473-9860-a3a68ddd3e3d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:05.935378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:05.940215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:05.940215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:05.940215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] PATH_INFO: `/v3/projects/53aebc4cb2b248369514a26a8d2d4ea6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:05.940215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=53aebc4cb2b248369514a26a8d2d4ea6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:05.940215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:06.091727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53aebc4cb2b248369514a26a8d2d4ea6', 'request_id': 'req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.098024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53aebc4cb2b248369514a26a8d2d4ea6', 'request_id': 'req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.104425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53aebc4cb2b248369514a26a8d2d4ea6', 'request_id': 'req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.171020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] The token cache is being invalidate because project 53aebc4cb2b248369514a26a8d2d4ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:35:06.176618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.176618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.176618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1103/2220] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:35:05 2022] DELETE /identity/v3/projects/53aebc4cb2b248369514a26a8d2d4ea6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:35:06.178006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbdc12-2f6f-47d8-a6b0-f57b535eb6b9 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:06.188427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbdc12-2f6f-47d8-a6b0-f57b535eb6b9 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:06.249234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:06.249871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:06.250149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:06.250423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] PATH_INFO: `/v3/users/aac21d9d9b5248b7904259ff98820cbd` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:06.251581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac21d9d9b5248b7904259ff98820cbd)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:06.253535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:06.277793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:06.423788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac21d9d9b5248b7904259ff98820cbd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.424533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac21d9d9b5248b7904259ff98820cbd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.440800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac21d9d9b5248b7904259ff98820cbd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.447709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac21d9d9b5248b7904259ff98820cbd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.448437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45d1d941-7a7b-4991-bff0-f100b638b853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac21d9d9b5248b7904259ff98820cbd'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:06.523660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1118/2221] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:35:06 2022] DELETE /identity/v3/users/aac21d9d9b5248b7904259ff98820cbd => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:06.913733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:06.915255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:06.929539 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5d77b-5ea3-497d-aa9b-b497dd82a1cf admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:06.987150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:06.987784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:06.988063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:06.988370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] PATH_INFO: `/v3/projects/65adfb8b2a9f42b7b03785aa4c1da79b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:06.989479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65adfb8b2a9f42b7b03785aa4c1da79b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:06.991471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:07.003395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:07.078883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65adfb8b2a9f42b7b03785aa4c1da79b', 'request_id': 'req-4352afe5-924f-47df-851c-41a7d198817a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:07.089842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65adfb8b2a9f42b7b03785aa4c1da79b', 'request_id': 'req-4352afe5-924f-47df-851c-41a7d198817a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:07.096510 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65adfb8b2a9f42b7b03785aa4c1da79b', 'request_id': 'req-4352afe5-924f-47df-851c-41a7d198817a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:07.192388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] The token cache is being invalidate because project 65adfb8b2a9f42b7b03785aa4c1da79b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:35:07.192648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:07.193306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4352afe5-924f-47df-851c-41a7d198817a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:07.194985 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1104/2222] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:35:06 2022] DELETE /identity/v3/projects/65adfb8b2a9f42b7b03785aa4c1da79b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:07.200611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd86ac5-989a-4a29-9e71-aab4bb999891 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.201106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd86ac5-989a-4a29-9e71-aab4bb999891 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.201213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd86ac5-989a-4a29-9e71-aab4bb999891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.222044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2cd86ac5-989a-4a29-9e71-aab4bb999891 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:07.223270 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2cd86ac5-989a-4a29-9e71-aab4bb999891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:07.282840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1119/2223] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:07.290581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf862cf3-c80a-4759-9868-26c3fbe90dd3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.290977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf862cf3-c80a-4759-9868-26c3fbe90dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.291259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf862cf3-c80a-4759-9868-26c3fbe90dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.311846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bf862cf3-c80a-4759-9868-26c3fbe90dd3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:07.318406 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf862cf3-c80a-4759-9868-26c3fbe90dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:07.324101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1105/2224] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:07.326305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd86ac5-989a-4a29-9e71-aab4bb999891 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:07.335961 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd86ac5-989a-4a29-9e71-aab4bb999891 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:07.340632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:07.347240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.347240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.347240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.347240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:35:07.347240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:07.347240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:07.357156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1120/2225] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:35:07.361530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf862cf3-c80a-4759-9868-26c3fbe90dd3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:07.370795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf862cf3-c80a-4759-9868-26c3fbe90dd3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:07.376030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8c78d-0571-4e6c-bffd-34011408b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:07.376654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8c78d-0571-4e6c-bffd-34011408b730 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.376933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8c78d-0571-4e6c-bffd-34011408b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.377205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8c78d-0571-4e6c-bffd-34011408b730 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.377826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8c78d-0571-4e6c-bffd-34011408b730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:07.379740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8c78d-0571-4e6c-bffd-34011408b730 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:07.485273 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1106/2226] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:35:07 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:07.486739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:07.498756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c036e06-0c9e-45c7-ac5f-251b4421966c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:07.563815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e31aa-0a1e-43e9-b035-750b7e65dfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:07.564451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e31aa-0a1e-43e9-b035-750b7e65dfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.564731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e31aa-0a1e-43e9-b035-750b7e65dfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.565005 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e31aa-0a1e-43e9-b035-750b7e65dfe3 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.565645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e31aa-0a1e-43e9-b035-750b7e65dfe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:07.567548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e31aa-0a1e-43e9-b035-750b7e65dfe3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:07.638383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1121/2227] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:35:07 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:07.642515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8c78d-0571-4e6c-bffd-34011408b730 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:07.652667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8c78d-0571-4e6c-bffd-34011408b730 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:07.657302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e35a0-433f-477d-a62e-82b2e4ccad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:07.657887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e35a0-433f-477d-a62e-82b2e4ccad7f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.658166 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e35a0-433f-477d-a62e-82b2e4ccad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.658434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e35a0-433f-477d-a62e-82b2e4ccad7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.659067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e35a0-433f-477d-a62e-82b2e4ccad7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:07.660710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e35a0-433f-477d-a62e-82b2e4ccad7f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:07.671455 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1107/2228] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:35:07.675241 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e31aa-0a1e-43e9-b035-750b7e65dfe3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:07.685991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e31aa-0a1e-43e9-b035-750b7e65dfe3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:07.690519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e34e4-9aa1-4706-be16-5a0072b269d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:07.691135 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e34e4-9aa1-4706-be16-5a0072b269d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.691396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e34e4-9aa1-4706-be16-5a0072b269d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.691664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e34e4-9aa1-4706-be16-5a0072b269d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.692365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e34e4-9aa1-4706-be16-5a0072b269d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:07.693959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e34e4-9aa1-4706-be16-5a0072b269d1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:07.704650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1122/2229] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:35:07.708688 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e35a0-433f-477d-a62e-82b2e4ccad7f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:07.718375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e35a0-433f-477d-a62e-82b2e4ccad7f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:07.722942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c39a60-96cc-4c56-8c7d-0e000494ef0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:07.723576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c39a60-96cc-4c56-8c7d-0e000494ef0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.723853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c39a60-96cc-4c56-8c7d-0e000494ef0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.724126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c39a60-96cc-4c56-8c7d-0e000494ef0e admin admin] PATH_INFO: `/v3/projects/9a78ea8a648d4c1da70dfd55a0c3dc05/users/7f97327f9fc848fea2c432ae30f4c9df/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.739209 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c39a60-96cc-4c56-8c7d-0e000494ef0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a78ea8a648d4c1da70dfd55a0c3dc05, user_id=7f97327f9fc848fea2c432ae30f4c9df, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:07.741874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c39a60-96cc-4c56-8c7d-0e000494ef0e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:07.808988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1108/2230] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:35:07 2022] PUT /identity/v3/projects/9a78ea8a648d4c1da70dfd55a0c3dc05/users/7f97327f9fc848fea2c432ae30f4c9df/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:07.821710 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e6b05-be0f-4fa1-9521-1897a95113b2 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.821999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e6b05-be0f-4fa1-9521-1897a95113b2 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.822305 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e6b05-be0f-4fa1-9521-1897a95113b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.835811 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:07.850332 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c01e6b05-be0f-4fa1-9521-1897a95113b2 None None] MFA Rules not processed for user `7f97327f9fc848fea2c432ae30f4c9df`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:07.851523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c01e6b05-be0f-4fa1-9521-1897a95113b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:07.913646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1123/2231] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:07.918799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c39a60-96cc-4c56-8c7d-0e000494ef0e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:07.978530 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c39a60-96cc-4c56-8c7d-0e000494ef0e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:07.984068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffc8be-56cd-4518-9184-57a48d5b83f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:07.984710 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffc8be-56cd-4518-9184-57a48d5b83f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:07.984987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffc8be-56cd-4518-9184-57a48d5b83f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:07.985259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffc8be-56cd-4518-9184-57a48d5b83f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:07.985878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffc8be-56cd-4518-9184-57a48d5b83f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:07.987775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffc8be-56cd-4518-9184-57a48d5b83f7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:08.091092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1109/2232] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:35:07 2022] POST /identity/v3/projects => generated 389 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:08.095107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e6b05-be0f-4fa1-9521-1897a95113b2 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:08.106832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e6b05-be0f-4fa1-9521-1897a95113b2 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:08.162009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e56f76-d062-4031-8360-166978f7437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:08.162632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56f76-d062-4031-8360-166978f7437a admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.162905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56f76-d062-4031-8360-166978f7437a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.163175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56f76-d062-4031-8360-166978f7437a admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.163800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e56f76-d062-4031-8360-166978f7437a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:08.165743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e56f76-d062-4031-8360-166978f7437a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:08.249501 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1124/2233] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:35:08 2022] POST /identity/v3/users => generated 301 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:08.253232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffc8be-56cd-4518-9184-57a48d5b83f7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:08.264577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffc8be-56cd-4518-9184-57a48d5b83f7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:08.268916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176199b-7652-40b3-8663-55bd3d2a0a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:08.269514 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176199b-7652-40b3-8663-55bd3d2a0a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.269790 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176199b-7652-40b3-8663-55bd3d2a0a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.270064 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176199b-7652-40b3-8663-55bd3d2a0a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.270704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176199b-7652-40b3-8663-55bd3d2a0a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:08.272415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176199b-7652-40b3-8663-55bd3d2a0a6d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:08.284725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1110/2234] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:35:08.289050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e56f76-d062-4031-8360-166978f7437a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:08.299729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e56f76-d062-4031-8360-166978f7437a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:08.304521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b105aa-5698-474d-b001-cc82161457f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:08.305174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b105aa-5698-474d-b001-cc82161457f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.305483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b105aa-5698-474d-b001-cc82161457f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.305757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b105aa-5698-474d-b001-cc82161457f9 admin admin] PATH_INFO: `/v3/projects/af022fd88ac846a69d9250aa78547ded/users/b262739576ef48aaa0dec4fc24d6a89d/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.321399 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b105aa-5698-474d-b001-cc82161457f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af022fd88ac846a69d9250aa78547ded, user_id=b262739576ef48aaa0dec4fc24d6a89d, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:08.324120 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b105aa-5698-474d-b001-cc82161457f9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:08.399484 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1125/2235] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:35:08 2022] PUT /identity/v3/projects/af022fd88ac846a69d9250aa78547ded/users/b262739576ef48aaa0dec4fc24d6a89d/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:08.403382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176199b-7652-40b3-8663-55bd3d2a0a6d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:08.413612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176199b-7652-40b3-8663-55bd3d2a0a6d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:08.421905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:08.470520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02285d49-67ee-43b8-ac52-b22a2ce32cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:08.471140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02285d49-67ee-43b8-ac52-b22a2ce32cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.471415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02285d49-67ee-43b8-ac52-b22a2ce32cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.471680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02285d49-67ee-43b8-ac52-b22a2ce32cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.472325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02285d49-67ee-43b8-ac52-b22a2ce32cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:08.473904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02285d49-67ee-43b8-ac52-b22a2ce32cf4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:08.483750 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1111/2236] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:35:08.487813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b105aa-5698-474d-b001-cc82161457f9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:08.498427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b105aa-5698-474d-b001-cc82161457f9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:08.503024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:08.503601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.503874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.504140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] PATH_INFO: `/v3/projects/af022fd88ac846a69d9250aa78547ded/users/b262739576ef48aaa0dec4fc24d6a89d/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.506041 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af022fd88ac846a69d9250aa78547ded, user_id=b262739576ef48aaa0dec4fc24d6a89d, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:08.508672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:08.518790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1126/2237] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:35:08 2022] PUT /identity/v3/projects/af022fd88ac846a69d9250aa78547ded/users/b262739576ef48aaa0dec4fc24d6a89d/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:08.584376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c124-38b1-4e2e-a216-13b01da7c74b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.584657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c124-38b1-4e2e-a216-13b01da7c74b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.584930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6c124-38b1-4e2e-a216-13b01da7c74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.613732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-61a6c124-38b1-4e2e-a216-13b01da7c74b None None] MFA Rules not processed for user `b262739576ef48aaa0dec4fc24d6a89d`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:08.614926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-61a6c124-38b1-4e2e-a216-13b01da7c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:08.686909 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1112/2238] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:08.698048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:08.699752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:08.759526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:08.764139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbec62-2540-49d2-8481-86ee9e5c25e6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:08.767798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88bc90-21aa-460a-838b-bcb3465dc7e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:08.768590 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88bc90-21aa-460a-838b-bcb3465dc7e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:08.768870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88bc90-21aa-460a-838b-bcb3465dc7e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:08.769327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88bc90-21aa-460a-838b-bcb3465dc7e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:08.779999 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88bc90-21aa-460a-838b-bcb3465dc7e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:08.781742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88bc90-21aa-460a-838b-bcb3465dc7e6 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:08.794105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1127/2239] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:08.942912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c124-38b1-4e2e-a216-13b01da7c74b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:09.005189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6c124-38b1-4e2e-a216-13b01da7c74b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:09.009749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:09.010307 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.010586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.010865 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] PATH_INFO: `/v3/users/7f97327f9fc848fea2c432ae30f4c9df` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:09.012502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f97327f9fc848fea2c432ae30f4c9df)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:09.014531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:09.226160 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:09.234982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678511b9-4a1f-4883-bd3d-bd2c44b3dd54 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.235272 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678511b9-4a1f-4883-bd3d-bd2c44b3dd54 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.235540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678511b9-4a1f-4883-bd3d-bd2c44b3dd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:09.236119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f97327f9fc848fea2c432ae30f4c9df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:09.243431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f97327f9fc848fea2c432ae30f4c9df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:09.255098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-678511b9-4a1f-4883-bd3d-bd2c44b3dd54 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:09.255829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f97327f9fc848fea2c432ae30f4c9df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:09.257729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-678511b9-4a1f-4883-bd3d-bd2c44b3dd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.312691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f97327f9fc848fea2c432ae30f4c9df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:09.313327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f97327f9fc848fea2c432ae30f4c9df'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:09.315764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1128/2240] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:35:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:35:09.319990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1113/2241] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:35:08 2022] DELETE /identity/v3/users/7f97327f9fc848fea2c432ae30f4c9df => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:09.325625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b419-5edb-47ca-9288-aa751678f48b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.325904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b419-5edb-47ca-9288-aa751678f48b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.326173 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53b419-5edb-47ca-9288-aa751678f48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:09.346200 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-ff53b419-5edb-47ca-9288-aa751678f48b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:09.347382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ff53b419-5edb-47ca-9288-aa751678f48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.404891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1129/2242] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:09.414485 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:09.415860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.451344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53b419-5edb-47ca-9288-aa751678f48b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:09.452838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ff53b419-5edb-47ca-9288-aa751678f48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.479416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6be9d-a149-4657-9bb8-a5e2bb2fa0f6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:09.483580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:09.484194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.484520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.484798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:09.495237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:09.497982 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:09.508574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1114/2243] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:09.533685 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53b419-5edb-47ca-9288-aa751678f48b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:09.540078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:09.541427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.541750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.542226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:09.544238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.594648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:09.617836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:09.619627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:09.628396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1130/2244] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:35:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:35:09.945441 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:09.949511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:09.978700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd255e4-055c-4d48-bdb5-46bb367b6cee service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:09.995623 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:09.999420 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:09.999704 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:09.999983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] PATH_INFO: `/v3/projects/9a78ea8a648d4c1da70dfd55a0c3dc05` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:10.001781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a78ea8a648d4c1da70dfd55a0c3dc05)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:10.007051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:10.135615 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a78ea8a648d4c1da70dfd55a0c3dc05', 'request_id': 'req-d6feacd9-0d1c-44ed-896c-ce6356970a1a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.145121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a78ea8a648d4c1da70dfd55a0c3dc05', 'request_id': 'req-d6feacd9-0d1c-44ed-896c-ce6356970a1a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.153210 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a78ea8a648d4c1da70dfd55a0c3dc05', 'request_id': 'req-d6feacd9-0d1c-44ed-896c-ce6356970a1a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.245407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] The token cache is being invalidate because project 9a78ea8a648d4c1da70dfd55a0c3dc05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:35:10.245811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.246723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.251566 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1115/2245] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:35:09 2022] DELETE /identity/v3/projects/9a78ea8a648d4c1da70dfd55a0c3dc05 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:35:10.253205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:10.264849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad237968-5495-4b28-bc19-ad955b72c890 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:10.344031 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:10.344885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:10.345354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:10.345757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] PATH_INFO: `/v3/users/b262739576ef48aaa0dec4fc24d6a89d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:10.347066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b262739576ef48aaa0dec4fc24d6a89d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:10.349392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:10.542349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b262739576ef48aaa0dec4fc24d6a89d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.543062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b262739576ef48aaa0dec4fc24d6a89d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.556189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b262739576ef48aaa0dec4fc24d6a89d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.563822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b262739576ef48aaa0dec4fc24d6a89d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.564567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0ef1d517-b923-4cf7-8ef7-06f45dcb6689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b262739576ef48aaa0dec4fc24d6a89d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:10.642064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1131/2246] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:35:10 2022] DELETE /identity/v3/users/b262739576ef48aaa0dec4fc24d6a89d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:11.121787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:11.124051 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:11.153285 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feacd9-0d1c-44ed-896c-ce6356970a1a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:11.226475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:11.269108 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:11.270092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:11.270644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:11.271084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] PATH_INFO: `/v3/projects/af022fd88ac846a69d9250aa78547ded` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:11.272574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af022fd88ac846a69d9250aa78547ded)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:11.274943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:11.400129 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af022fd88ac846a69d9250aa78547ded', 'request_id': 'req-5742e3c3-7b6c-4a02-866b-ab30e842a256'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:11.406425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af022fd88ac846a69d9250aa78547ded', 'request_id': 'req-5742e3c3-7b6c-4a02-866b-ab30e842a256'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:11.414381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af022fd88ac846a69d9250aa78547ded', 'request_id': 'req-5742e3c3-7b6c-4a02-866b-ab30e842a256'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:11.514105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] The token cache is being invalidate because project af022fd88ac846a69d9250aa78547ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:35:11.514495 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:11.515247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5742e3c3-7b6c-4a02-866b-ab30e842a256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:11.520207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1116/2247] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:35:11 2022] DELETE /identity/v3/projects/af022fd88ac846a69d9250aa78547ded => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:11.522464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf74b-1db1-41a8-8610-f3468b80e7fa None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:11.525264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf74b-1db1-41a8-8610-f3468b80e7fa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:11.525264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf74b-1db1-41a8-8610-f3468b80e7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:11.552358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-91fbf74b-1db1-41a8-8610-f3468b80e7fa None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:11.553737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-91fbf74b-1db1-41a8-8610-f3468b80e7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:11.611992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:11.621735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1132/2248] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:11.629682 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03097d-9626-40a9-8b56-93d3e1fa5d41 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:11.629971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03097d-9626-40a9-8b56-93d3e1fa5d41 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:11.630245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03097d-9626-40a9-8b56-93d3e1fa5d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:11.651433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cc03097d-9626-40a9-8b56-93d3e1fa5d41 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:11.653428 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cc03097d-9626-40a9-8b56-93d3e1fa5d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:11.662379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1117/2249] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:11.666861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf74b-1db1-41a8-8610-f3468b80e7fa None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:11.679296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf74b-1db1-41a8-8610-f3468b80e7fa None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:11.684704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:11.685370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:11.689248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:11.689248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:11.689248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:35:11.689248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:11.689248 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:11.701638 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1133/2250] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:35:11.705668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03097d-9626-40a9-8b56-93d3e1fa5d41 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:11.716065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03097d-9626-40a9-8b56-93d3e1fa5d41 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:11.722176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1505b32-15dd-48ef-b8d2-04c81ca57d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:11.722783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1505b32-15dd-48ef-b8d2-04c81ca57d66 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:11.723059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1505b32-15dd-48ef-b8d2-04c81ca57d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:11.723335 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1505b32-15dd-48ef-b8d2-04c81ca57d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:11.723963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1505b32-15dd-48ef-b8d2-04c81ca57d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:11.725933 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1505b32-15dd-48ef-b8d2-04c81ca57d66 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:11.801655 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1118/2251] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:35:11 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:35:11.806042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:11.816493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1c56e-72a5-4bcc-b9ac-abb80f182e19 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:11.886304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937aea3-df7c-4a14-9ca1-251f70db2558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:11.887551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937aea3-df7c-4a14-9ca1-251f70db2558 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:11.887826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937aea3-df7c-4a14-9ca1-251f70db2558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:11.888099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937aea3-df7c-4a14-9ca1-251f70db2558 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:11.890056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937aea3-df7c-4a14-9ca1-251f70db2558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:11.891973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937aea3-df7c-4a14-9ca1-251f70db2558 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:11.988250 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1134/2252] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:35:11 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:11.992401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1505b32-15dd-48ef-b8d2-04c81ca57d66 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:12.005809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1505b32-15dd-48ef-b8d2-04c81ca57d66 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:12.010715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9a67-b569-4836-b3bc-ac2856a2d470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:12.012053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9a67-b569-4836-b3bc-ac2856a2d470 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.012752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9a67-b569-4836-b3bc-ac2856a2d470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.012874 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e9a67-b569-4836-b3bc-ac2856a2d470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.013427 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e9a67-b569-4836-b3bc-ac2856a2d470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:12.015642 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e9a67-b569-4836-b3bc-ac2856a2d470 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:12.030292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1119/2253] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:35:12.034469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937aea3-df7c-4a14-9ca1-251f70db2558 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:12.049591 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937aea3-df7c-4a14-9ca1-251f70db2558 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:12.055784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b3305-2c53-4edc-b75e-c9deac6028a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:12.056348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3305-2c53-4edc-b75e-c9deac6028a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.056655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3305-2c53-4edc-b75e-c9deac6028a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.056930 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3305-2c53-4edc-b75e-c9deac6028a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.057579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b3305-2c53-4edc-b75e-c9deac6028a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:12.059163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b3305-2c53-4edc-b75e-c9deac6028a6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:12.074689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1135/2254] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:35:12.076235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9a67-b569-4836-b3bc-ac2856a2d470 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:12.092110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e9a67-b569-4836-b3bc-ac2856a2d470 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:12.098361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44b48a-5d58-4eae-b22f-6030b1aea983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:12.099096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44b48a-5d58-4eae-b22f-6030b1aea983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.099442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44b48a-5d58-4eae-b22f-6030b1aea983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.099779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44b48a-5d58-4eae-b22f-6030b1aea983 admin admin] PATH_INFO: `/v3/projects/1536271a0509462ab260ae68bf3b3364/users/29df65b5e05a4eeabde6862aa311d1a6/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.118867 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44b48a-5d58-4eae-b22f-6030b1aea983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1536271a0509462ab260ae68bf3b3364, user_id=29df65b5e05a4eeabde6862aa311d1a6, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:12.121686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44b48a-5d58-4eae-b22f-6030b1aea983 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:12.200065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1120/2255] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:35:12 2022] PUT /identity/v3/projects/1536271a0509462ab260ae68bf3b3364/users/29df65b5e05a4eeabde6862aa311d1a6/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:12.212185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4a303-b0ba-4802-8826-f53d59d48531 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.212500 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4a303-b0ba-4802-8826-f53d59d48531 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.212775 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4a303-b0ba-4802-8826-f53d59d48531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.255058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-06d4a303-b0ba-4802-8826-f53d59d48531 None None] MFA Rules not processed for user `29df65b5e05a4eeabde6862aa311d1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:12.256324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-06d4a303-b0ba-4802-8826-f53d59d48531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:12.335485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1136/2256] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:12.339823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44b48a-5d58-4eae-b22f-6030b1aea983 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:12.451386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44b48a-5d58-4eae-b22f-6030b1aea983 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:12.469153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a75937-61c2-4041-8328-49a86fc4e523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:12.469910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a75937-61c2-4041-8328-49a86fc4e523 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.470034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a75937-61c2-4041-8328-49a86fc4e523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.470283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a75937-61c2-4041-8328-49a86fc4e523 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.470934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a75937-61c2-4041-8328-49a86fc4e523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:12.472879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a75937-61c2-4041-8328-49a86fc4e523 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:12.485724 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:12.595013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1121/2257] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:35:12 2022] POST /identity/v3/projects => generated 379 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:12.596594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4a303-b0ba-4802-8826-f53d59d48531 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:12.638515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4a303-b0ba-4802-8826-f53d59d48531 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:12.698648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:12.719546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facff44e-45bb-4154-b3bc-274a20b471fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:12.720187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facff44e-45bb-4154-b3bc-274a20b471fd admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.720494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facff44e-45bb-4154-b3bc-274a20b471fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.720896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facff44e-45bb-4154-b3bc-274a20b471fd admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.721401 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facff44e-45bb-4154-b3bc-274a20b471fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:12.723611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facff44e-45bb-4154-b3bc-274a20b471fd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:12.811707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1137/2258] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:35:12 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:12.815746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a75937-61c2-4041-8328-49a86fc4e523 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:12.828135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a75937-61c2-4041-8328-49a86fc4e523 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:12.833327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186db68-ddc2-42ec-918b-a52657daf40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:12.833958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186db68-ddc2-42ec-918b-a52657daf40a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.834238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186db68-ddc2-42ec-918b-a52657daf40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.834507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186db68-ddc2-42ec-918b-a52657daf40a admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.835320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186db68-ddc2-42ec-918b-a52657daf40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:12.837244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186db68-ddc2-42ec-918b-a52657daf40a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:12.860410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1122/2259] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:35:12.865516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facff44e-45bb-4154-b3bc-274a20b471fd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:12.876984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facff44e-45bb-4154-b3bc-274a20b471fd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:12.881959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9fde-2713-4825-b654-91ef9b0b26d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:12.882688 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9fde-2713-4825-b654-91ef9b0b26d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.883048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9fde-2713-4825-b654-91ef9b0b26d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.883392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9fde-2713-4825-b654-91ef9b0b26d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.884127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9fde-2713-4825-b654-91ef9b0b26d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:12.888339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9fde-2713-4825-b654-91ef9b0b26d9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:12.900972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1138/2260] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:35:12.902310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186db68-ddc2-42ec-918b-a52657daf40a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:12.912878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186db68-ddc2-42ec-918b-a52657daf40a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:12.920597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c19fa-0edb-45ef-a12f-67e83b8419fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:12.921505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c19fa-0edb-45ef-a12f-67e83b8419fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:12.921781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c19fa-0edb-45ef-a12f-67e83b8419fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:12.922072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c19fa-0edb-45ef-a12f-67e83b8419fc admin admin] PATH_INFO: `/v3/projects/9b8bc9b9a3084b2594b88dbaef600b61/users/9062910214c04ba5b35f393718cc5b6d/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:12.942454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c19fa-0edb-45ef-a12f-67e83b8419fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8bc9b9a3084b2594b88dbaef600b61, user_id=9062910214c04ba5b35f393718cc5b6d, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:12.945395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c19fa-0edb-45ef-a12f-67e83b8419fc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:13.348512 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1123/2261] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:35:12 2022] PUT /identity/v3/projects/9b8bc9b9a3084b2594b88dbaef600b61/users/9062910214c04ba5b35f393718cc5b6d/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:35:13.359371 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be50e02-df08-4bc8-99a4-65090e38695f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:13.359667 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be50e02-df08-4bc8-99a4-65090e38695f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:13.359971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be50e02-df08-4bc8-99a4-65090e38695f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:13.387783 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-4be50e02-df08-4bc8-99a4-65090e38695f None None] MFA Rules not processed for user `9062910214c04ba5b35f393718cc5b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:13.388981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4be50e02-df08-4bc8-99a4-65090e38695f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:13.451434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1139/2262] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:13.456000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c19fa-0edb-45ef-a12f-67e83b8419fc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:13.517013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c19fa-0edb-45ef-a12f-67e83b8419fc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:13.521893 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01db799-0796-4abb-87c3-4b805737d6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:13.522620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db799-0796-4abb-87c3-4b805737d6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:13.522829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db799-0796-4abb-87c3-4b805737d6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:13.523103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01db799-0796-4abb-87c3-4b805737d6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:13.524684 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01db799-0796-4abb-87c3-4b805737d6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:13.526726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01db799-0796-4abb-87c3-4b805737d6e2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:13.603643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1124/2263] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:35:13 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:13.605499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be50e02-df08-4bc8-99a4-65090e38695f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:13.616385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be50e02-df08-4bc8-99a4-65090e38695f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:13.723928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2665b-3888-42ad-b4e3-89e7aed1090c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:13.724583 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2665b-3888-42ad-b4e3-89e7aed1090c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:13.724928 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2665b-3888-42ad-b4e3-89e7aed1090c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:13.725204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2665b-3888-42ad-b4e3-89e7aed1090c admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:13.729028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2665b-3888-42ad-b4e3-89e7aed1090c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:13.732319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2665b-3888-42ad-b4e3-89e7aed1090c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:13.810670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1140/2264] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:35:13 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:35:13.815029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01db799-0796-4abb-87c3-4b805737d6e2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:13.827894 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01db799-0796-4abb-87c3-4b805737d6e2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:13.834382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccac75-ca5a-420b-aad1-0f515d036ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:13.837053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccac75-ca5a-420b-aad1-0f515d036ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:13.837053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccac75-ca5a-420b-aad1-0f515d036ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:13.837053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccac75-ca5a-420b-aad1-0f515d036ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:13.837408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccac75-ca5a-420b-aad1-0f515d036ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:13.840625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccac75-ca5a-420b-aad1-0f515d036ff7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:13.860835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2665b-3888-42ad-b4e3-89e7aed1090c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:13.861761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1125/2265] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:35:13.880214 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2665b-3888-42ad-b4e3-89e7aed1090c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:13.901848 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9bd07-90d1-406b-b0af-128f697301c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:13.902616 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9bd07-90d1-406b-b0af-128f697301c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:13.902965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9bd07-90d1-406b-b0af-128f697301c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:13.903311 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9bd07-90d1-406b-b0af-128f697301c0 admin admin] PATH_INFO: `/v3/projects/7ab19afed323468794e63f4a5cd8dcb7/users/bdd64a374c8e4a0590d8a64f22b55a08/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:13.929536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9bd07-90d1-406b-b0af-128f697301c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab19afed323468794e63f4a5cd8dcb7, user_id=bdd64a374c8e4a0590d8a64f22b55a08, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:13.932461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9bd07-90d1-406b-b0af-128f697301c0 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:14.004229 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1141/2266] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:35:13 2022] PUT /identity/v3/projects/7ab19afed323468794e63f4a5cd8dcb7/users/bdd64a374c8e4a0590d8a64f22b55a08/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:14.005869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccac75-ca5a-420b-aad1-0f515d036ff7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:14.021694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccac75-ca5a-420b-aad1-0f515d036ff7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:14.131186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f498a-8559-4d68-81f8-56f57dbdf8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:14.131912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f498a-8559-4d68-81f8-56f57dbdf8ce admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:14.133553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f498a-8559-4d68-81f8-56f57dbdf8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:14.133553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f498a-8559-4d68-81f8-56f57dbdf8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:14.133553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f498a-8559-4d68-81f8-56f57dbdf8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:14.135152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f498a-8559-4d68-81f8-56f57dbdf8ce admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:14.162305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1126/2267] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:35:14.162517 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9bd07-90d1-406b-b0af-128f697301c0 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:14.174743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9bd07-90d1-406b-b0af-128f697301c0 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:14.180254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:14.186375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:14.186375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:14.186375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] PATH_INFO: `/v3/projects/7ab19afed323468794e63f4a5cd8dcb7/users/bdd64a374c8e4a0590d8a64f22b55a08/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:14.186375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab19afed323468794e63f4a5cd8dcb7, user_id=bdd64a374c8e4a0590d8a64f22b55a08, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:14.186913 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:14.219694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1142/2268] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:35:14 2022] PUT /identity/v3/projects/7ab19afed323468794e63f4a5cd8dcb7/users/bdd64a374c8e4a0590d8a64f22b55a08/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:14.230756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491495-c75a-4fce-9e35-c39821d29c51 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:14.231185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491495-c75a-4fce-9e35-c39821d29c51 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:14.231185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491495-c75a-4fce-9e35-c39821d29c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:14.271402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c0491495-c75a-4fce-9e35-c39821d29c51 None None] MFA Rules not processed for user `bdd64a374c8e4a0590d8a64f22b55a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:14.273080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c0491495-c75a-4fce-9e35-c39821d29c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:14.278488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:14.386997 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1127/2269] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:14.399601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:14.401601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:14.418378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:14.467393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560be28e-21f2-46e8-bdb3-a4562aef4560 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:14.471355 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:14.472027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:14.472414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:14.472753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:14.498904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:14.500487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:14.511432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1143/2270] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:14.624208 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491495-c75a-4fce-9e35-c39821d29c51 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:14.625859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c0491495-c75a-4fce-9e35-c39821d29c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:14.647652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491495-c75a-4fce-9e35-c39821d29c51 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:14.651841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9f9e-4dae-496e-8489-e1142d629e2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:14.653099 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9f9e-4dae-496e-8489-e1142d629e2e service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:14.653410 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9f9e-4dae-496e-8489-e1142d629e2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:14.653687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9f9e-4dae-496e-8489-e1142d629e2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:14.737712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a9f9e-4dae-496e-8489-e1142d629e2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:14.739268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a9f9e-4dae-496e-8489-e1142d629e2e service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:14.753476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1128/2271] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:15.682897 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:15.685926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:15.761225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec68e4-a31b-40ca-a180-7eddd68a27de service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:15.765314 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:15.766065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:15.766673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:15.766793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:15.768943 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:15.854972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:15.857082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:15.869369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1144/2272] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:16.014195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9f9e-4dae-496e-8489-e1142d629e2e service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:16.024756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9f9e-4dae-496e-8489-e1142d629e2e service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:16.029150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:16.029756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:16.030039 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:16.030311 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:16.032048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:16.121513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:16.121513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:16.125734 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1129/2273] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:19.492630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:19.568689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:19.573185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d07a-69b7-4a74-b0f8-9af40743592d service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:19.579676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:19.580799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:19.581230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:19.581750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] PATH_INFO: `/v3/users/29df65b5e05a4eeabde6862aa311d1a6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:19.585416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29df65b5e05a4eeabde6862aa311d1a6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:19.587541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:19.785983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29df65b5e05a4eeabde6862aa311d1a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:19.786669 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29df65b5e05a4eeabde6862aa311d1a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:19.826411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29df65b5e05a4eeabde6862aa311d1a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:19.835792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29df65b5e05a4eeabde6862aa311d1a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:19.836551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-54193559-f497-436b-a59f-b30c6933f3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29df65b5e05a4eeabde6862aa311d1a6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:19.890063 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:19.892312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:19.906751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f23c5-74a1-4a19-9f12-6efd9968a28d service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:19.911011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:19.911011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:19.911011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:19.911011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:19.912742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:19.929425 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:19.930922 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:19.942640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1145/2274] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:35:19 2022] DELETE /identity/v3/users/29df65b5e05a4eeabde6862aa311d1a6 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:19.946916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a67e4f-0944-480b-bb50-5f45f824b818 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:19.947207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a67e4f-0944-480b-bb50-5f45f824b818 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:19.947480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a67e4f-0944-480b-bb50-5f45f824b818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:19.959314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1130/2275] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:35:19.978624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-50a67e4f-0944-480b-bb50-5f45f824b818 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:19.979833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-50a67e4f-0944-480b-bb50-5f45f824b818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:20.079576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1146/2276] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:20.091336 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:20.093364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:20.121834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e596-283f-4a9e-8704-d4cb3b90f9e6 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:20.150608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:20.187117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837dc44-5f72-491c-bd06-9b94ca04ab6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:20.187864 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837dc44-5f72-491c-bd06-9b94ca04ab6b service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:20.188537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837dc44-5f72-491c-bd06-9b94ca04ab6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:20.188812 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837dc44-5f72-491c-bd06-9b94ca04ab6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:20.199724 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837dc44-5f72-491c-bd06-9b94ca04ab6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:20.201546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837dc44-5f72-491c-bd06-9b94ca04ab6b service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:20.211866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1131/2277] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:20.776972 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a67e4f-0944-480b-bb50-5f45f824b818 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:20.778563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-50a67e4f-0944-480b-bb50-5f45f824b818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:20.790718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a67e4f-0944-480b-bb50-5f45f824b818 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:20.795549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:20.796188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:20.796490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:20.796772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] PATH_INFO: `/v3/projects/1536271a0509462ab260ae68bf3b3364` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:20.797890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1536271a0509462ab260ae68bf3b3364)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:20.799799 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:20.903435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1536271a0509462ab260ae68bf3b3364', 'request_id': 'req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:20.992627 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1536271a0509462ab260ae68bf3b3364', 'request_id': 'req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.000847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1536271a0509462ab260ae68bf3b3364', 'request_id': 'req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.009108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:21.013664 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] The token cache is being invalidate because project 1536271a0509462ab260ae68bf3b3364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:35:21.014066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.014809 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.016411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1147/2278] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:35:20 2022] DELETE /identity/v3/projects/1536271a0509462ab260ae68bf3b3364 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:21.021006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837dc44-5f72-491c-bd06-9b94ca04ab6b service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:21.032921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837dc44-5f72-491c-bd06-9b94ca04ab6b service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:21.110435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:21.111900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:21.112459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:21.112713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] PATH_INFO: `/v3/users/9062910214c04ba5b35f393718cc5b6d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:21.113980 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9062910214c04ba5b35f393718cc5b6d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:21.116021 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:21.315774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9062910214c04ba5b35f393718cc5b6d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.326678 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9062910214c04ba5b35f393718cc5b6d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.342433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9062910214c04ba5b35f393718cc5b6d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.415349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9062910214c04ba5b35f393718cc5b6d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.415904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9062910214c04ba5b35f393718cc5b6d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:21.423440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1132/2279] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:35:21 2022] DELETE /identity/v3/users/9062910214c04ba5b35f393718cc5b6d => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:21.862995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:21.864467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:21.878508 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3b9e1-1ddc-41bf-83c3-d7ff86cca969 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:21.942673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:21.942673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:21.942673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:21.942673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] PATH_INFO: `/v3/projects/9b8bc9b9a3084b2594b88dbaef600b61` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:21.943190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8bc9b9a3084b2594b88dbaef600b61)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:21.943190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:22.043485 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8bc9b9a3084b2594b88dbaef600b61', 'request_id': 'req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.089228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8bc9b9a3084b2594b88dbaef600b61', 'request_id': 'req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.096935 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8bc9b9a3084b2594b88dbaef600b61', 'request_id': 'req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.107602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] The token cache is being invalidate because project 9b8bc9b9a3084b2594b88dbaef600b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:35:22.107909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.108598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.110074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1148/2280] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:35:21 2022] DELETE /identity/v3/projects/9b8bc9b9a3084b2594b88dbaef600b61 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:22.114117 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:22.115480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:22.128852 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032e339-1f8f-45bb-b875-0b5b9e5d9e94 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:22.168396 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:22.184725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:22.185340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:22.185640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:22.185908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] PATH_INFO: `/v3/users/bdd64a374c8e4a0590d8a64f22b55a08` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:22.187000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd64a374c8e4a0590d8a64f22b55a08)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:22.189114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:22.371453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd64a374c8e4a0590d8a64f22b55a08'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.380569 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd64a374c8e4a0590d8a64f22b55a08'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.392869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd64a374c8e4a0590d8a64f22b55a08'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.455017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd64a374c8e4a0590d8a64f22b55a08'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.455657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7908c2bb-a129-4ed5-876d-c5b7614e9034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd64a374c8e4a0590d8a64f22b55a08'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:22.462261 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1133/2281] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:35:22 2022] DELETE /identity/v3/users/bdd64a374c8e4a0590d8a64f22b55a08 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:22.874059 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:22.875472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:22.898588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff9848-b71d-4eb7-9147-e7d1ab1370b1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:22.958752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:22.959561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:22.959840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:22.960109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] PATH_INFO: `/v3/projects/7ab19afed323468794e63f4a5cd8dcb7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:22.961237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab19afed323468794e63f4a5cd8dcb7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:22.963278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:23.067480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:23.080154 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab19afed323468794e63f4a5cd8dcb7', 'request_id': 'req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:23.141454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab19afed323468794e63f4a5cd8dcb7', 'request_id': 'req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:23.148973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab19afed323468794e63f4a5cd8dcb7', 'request_id': 'req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:23.161479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] The token cache is being invalidate because project 7ab19afed323468794e63f4a5cd8dcb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:35:23.161827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:23.162686 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7cd9e2f3-cb94-40cd-8119-fd63132f3e45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:35:23.164480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1149/2282] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:35:22 2022] DELETE /identity/v3/projects/7ab19afed323468794e63f4a5cd8dcb7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:23.171779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d5fc9-940d-4ac6-a8a4-cf162627a447 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.172353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d5fc9-940d-4ac6-a8a4-cf162627a447 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.172583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d5fc9-940d-4ac6-a8a4-cf162627a447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.193910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c22d5fc9-940d-4ac6-a8a4-cf162627a447 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:23.195253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c22d5fc9-940d-4ac6-a8a4-cf162627a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:23.259237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1134/2283] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:23.268786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779ae48-5a8f-4444-9499-7d8bfc771fa9 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.269075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779ae48-5a8f-4444-9499-7d8bfc771fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.269352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779ae48-5a8f-4444-9499-7d8bfc771fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.289524 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f779ae48-5a8f-4444-9499-7d8bfc771fa9 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:23.290723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f779ae48-5a8f-4444-9499-7d8bfc771fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:23.298994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1150/2284] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:23.303232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d5fc9-940d-4ac6-a8a4-cf162627a447 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:23.313050 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d5fc9-940d-4ac6-a8a4-cf162627a447 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:23.317703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:23.318299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.318599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.318871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.319515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:35:23.319821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:23.321430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:23.334404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1135/2285] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:35:23.338596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779ae48-5a8f-4444-9499-7d8bfc771fa9 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:23.348874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f779ae48-5a8f-4444-9499-7d8bfc771fa9 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:23.353582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f674b7b-bae0-4d15-8cfe-03331ad0d435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:23.354171 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f674b7b-bae0-4d15-8cfe-03331ad0d435 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.354445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f674b7b-bae0-4d15-8cfe-03331ad0d435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.354709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f674b7b-bae0-4d15-8cfe-03331ad0d435 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.355341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f674b7b-bae0-4d15-8cfe-03331ad0d435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:23.357506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f674b7b-bae0-4d15-8cfe-03331ad0d435 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:23.437712 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1151/2286] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:35:23 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:23.438897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:23.449319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95647073-af3a-430a-855f-d2972e266df6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:23.496015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:23.508947 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973870-b613-4908-949b-e5911e6497a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:23.509553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973870-b613-4908-949b-e5911e6497a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.509830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973870-b613-4908-949b-e5911e6497a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.510101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973870-b613-4908-949b-e5911e6497a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.510725 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973870-b613-4908-949b-e5911e6497a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:23.513236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973870-b613-4908-949b-e5911e6497a1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:23.638908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1136/2287] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:35:23 2022] POST /identity/v3/users => generated 282 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:35:23.643693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f674b7b-bae0-4d15-8cfe-03331ad0d435 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:23.653936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f674b7b-bae0-4d15-8cfe-03331ad0d435 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:23.658479 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f640b-5d73-44f1-b21b-348f47ca2794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:23.659072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f640b-5d73-44f1-b21b-348f47ca2794 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.659385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f640b-5d73-44f1-b21b-348f47ca2794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.659659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f640b-5d73-44f1-b21b-348f47ca2794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.660392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f640b-5d73-44f1-b21b-348f47ca2794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:23.661932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f640b-5d73-44f1-b21b-348f47ca2794 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:23.672257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1152/2288] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:35:23.676402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973870-b613-4908-949b-e5911e6497a1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:23.688592 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973870-b613-4908-949b-e5911e6497a1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:23.693286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62149cbc-d0e3-4171-90aa-dbbd648524fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:23.693882 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62149cbc-d0e3-4171-90aa-dbbd648524fe admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.694158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62149cbc-d0e3-4171-90aa-dbbd648524fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.694426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62149cbc-d0e3-4171-90aa-dbbd648524fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.695094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62149cbc-d0e3-4171-90aa-dbbd648524fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:23.696763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62149cbc-d0e3-4171-90aa-dbbd648524fe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:23.708560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1137/2289] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:35:23.712814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f640b-5d73-44f1-b21b-348f47ca2794 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:23.722864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f640b-5d73-44f1-b21b-348f47ca2794 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:23.727358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:23.727997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.728291 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.728579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] PATH_INFO: `/v3/projects/fad33c8add3e499caa13e45fc7af5a4f/users/535e7c58ae0c450dab23d8097531ee5b/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.742955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad33c8add3e499caa13e45fc7af5a4f, user_id=535e7c58ae0c450dab23d8097531ee5b, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:23.745645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:23.815884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1153/2290] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:35:23 2022] PUT /identity/v3/projects/fad33c8add3e499caa13e45fc7af5a4f/users/535e7c58ae0c450dab23d8097531ee5b/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:35:23.828176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5034a4-4fa9-4eaa-9d76-5a23520b0da8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.828492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5034a4-4fa9-4eaa-9d76-5a23520b0da8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.828765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5034a4-4fa9-4eaa-9d76-5a23520b0da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:23.855859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bf5034a4-4fa9-4eaa-9d76-5a23520b0da8 None None] MFA Rules not processed for user `535e7c58ae0c450dab23d8097531ee5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:23.857605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf5034a4-4fa9-4eaa-9d76-5a23520b0da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:23.919587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1138/2291] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:23.932675 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:23.934866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:23.990282 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31db4c-40a7-4c93-8735-b285cc8d1a0a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:23.993944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00373b6-7e3f-4eaf-bb47-bd1b2a275485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:23.994530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00373b6-7e3f-4eaf-bb47-bd1b2a275485 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:23.999445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00373b6-7e3f-4eaf-bb47-bd1b2a275485 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:23.999445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00373b6-7e3f-4eaf-bb47-bd1b2a275485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:24.004012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00373b6-7e3f-4eaf-bb47-bd1b2a275485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:24.005451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00373b6-7e3f-4eaf-bb47-bd1b2a275485 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:24.014246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1154/2292] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:28.375661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5034a4-4fa9-4eaa-9d76-5a23520b0da8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:28.377077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf5034a4-4fa9-4eaa-9d76-5a23520b0da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:28.437189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5034a4-4fa9-4eaa-9d76-5a23520b0da8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:28.440991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:28.441596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:28.441885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:28.442172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:28.444046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:28.516703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:28.518118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:28.522752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:28.529308 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1139/2293] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:35:28.536746 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00373b6-7e3f-4eaf-bb47-bd1b2a275485 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:28.546676 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00373b6-7e3f-4eaf-bb47-bd1b2a275485 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:28.550157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:28.550743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:28.551026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:28.551299 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:28.553307 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:28.614136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:28.633949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:28.635386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:28.653994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1155/2294] 10.222.0.48 () {60 vars in 1396 bytes} [Mon Oct 24 13:35:28 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:28.859938 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:28.861405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:28.918574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4d45c-ac6b-4ca3-ba17-f98486dfc888 service glance] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:28.923430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05b529-2d44-47b6-821a-f135a935864e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:28.924072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b529-2d44-47b6-821a-f135a935864e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:28.924383 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b529-2d44-47b6-821a-f135a935864e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:28.924662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05b529-2d44-47b6-821a-f135a935864e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:28.926394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eb05b529-2d44-47b6-821a-f135a935864e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:28.935883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05b529-2d44-47b6-821a-f135a935864e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:28.937320 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05b529-2d44-47b6-821a-f135a935864e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:28.945034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1140/2295] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:29.020160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:29.030062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b457fa-7a3a-4cb6-8a61-fd9be5a2b4b3 service glance] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:29.033946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:29.034546 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:29.034827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:29.035100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:29.036832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:29.050911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:29.052378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:29.063147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1156/2296] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:35:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:35:32.144146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b964f12-4559-42cd-99c1-1463c0b8d60e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:32.146846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b964f12-4559-42cd-99c1-1463c0b8d60e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:32.146846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b964f12-4559-42cd-99c1-1463c0b8d60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:32.164910 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3b964f12-4559-42cd-99c1-1463c0b8d60e None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:32.166114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3b964f12-4559-42cd-99c1-1463c0b8d60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:32.223292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1141/2297] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:35:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:32.235731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:32.237191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:32.306683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe306-c592-4500-892d-4263dd4ab4e3 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:32.311598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2a2b8-94e0-43dd-8102-75e659396b5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:32.312168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2a2b8-94e0-43dd-8102-75e659396b5c service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:32.312483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2a2b8-94e0-43dd-8102-75e659396b5c service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:32.312750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2a2b8-94e0-43dd-8102-75e659396b5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:32.320614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2a2b8-94e0-43dd-8102-75e659396b5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:32.322203 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2a2b8-94e0-43dd-8102-75e659396b5c service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:32.331158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1157/2298] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:35:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:35:42.215962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=52 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:42.327817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=52 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:35:54.372928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93324e9-e466-48d9-b622-9729f9868685 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:54.373525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93324e9-e466-48d9-b622-9729f9868685 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:54.373525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93324e9-e466-48d9-b622-9729f9868685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:54.424770 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d93324e9-e466-48d9-b622-9729f9868685 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:35:54.426008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d93324e9-e466-48d9-b622-9729f9868685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:35:54.443578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1142/2299] 10.222.0.48 () {58 vars in 967 bytes} [Mon Oct 24 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:35:54.456817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2a2b8-94e0-43dd-8102-75e659396b5c service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:35:54.480306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2a2b8-94e0-43dd-8102-75e659396b5c service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:35:54.485398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bd800-efe7-4b4d-aaed-c44b9eace4de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:35:54.486596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bd800-efe7-4b4d-aaed-c44b9eace4de service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:35:54.486876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bd800-efe7-4b4d-aaed-c44b9eace4de service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:35:54.487149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bd800-efe7-4b4d-aaed-c44b9eace4de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:35:54.495460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bd800-efe7-4b4d-aaed-c44b9eace4de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:35:54.497078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bd800-efe7-4b4d-aaed-c44b9eace4de service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:35:54.505633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1158/2300] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:35:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:36:04.440332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=18 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:04.502359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:14.516908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93324e9-e466-48d9-b622-9729f9868685 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:14.580025 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93324e9-e466-48d9-b622-9729f9868685 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:14.585126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:14.585743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:14.586015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:14.586282 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] PATH_INFO: `/v3/users/535e7c58ae0c450dab23d8097531ee5b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:14.587387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=535e7c58ae0c450dab23d8097531ee5b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:14.589302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:14.771059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535e7c58ae0c450dab23d8097531ee5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:14.778003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535e7c58ae0c450dab23d8097531ee5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:14.790354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535e7c58ae0c450dab23d8097531ee5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:15.008753 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535e7c58ae0c450dab23d8097531ee5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:15.009387 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535e7c58ae0c450dab23d8097531ee5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:15.015920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1143/2301] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:36:14 2022] DELETE /identity/v3/users/535e7c58ae0c450dab23d8097531ee5b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:36:15.020891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209d83a-e33a-4fe6-bcab-56836f11e25a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:15.021385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209d83a-e33a-4fe6-bcab-56836f11e25a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:15.021492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209d83a-e33a-4fe6-bcab-56836f11e25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:15.040486 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5209d83a-e33a-4fe6-bcab-56836f11e25a None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:15.047797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5209d83a-e33a-4fe6-bcab-56836f11e25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:15.102587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1159/2302] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:15.112480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:15.113957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:15.171301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d204307-55a1-4c69-97de-9af1596583b2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:15.174918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c2044-62e2-4ba1-bcd9-a000ea0cf71c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:15.175550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c2044-62e2-4ba1-bcd9-a000ea0cf71c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:15.175832 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c2044-62e2-4ba1-bcd9-a000ea0cf71c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:15.176106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c2044-62e2-4ba1-bcd9-a000ea0cf71c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:15.184961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c2044-62e2-4ba1-bcd9-a000ea0cf71c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:15.186384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c2044-62e2-4ba1-bcd9-a000ea0cf71c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:15.196141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1144/2303] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:15.742331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209d83a-e33a-4fe6-bcab-56836f11e25a None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:15.744066 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5209d83a-e33a-4fe6-bcab-56836f11e25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:15.757661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209d83a-e33a-4fe6-bcab-56836f11e25a None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:15.762323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:15.762955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:15.763233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:15.763532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] PATH_INFO: `/v3/projects/fad33c8add3e499caa13e45fc7af5a4f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:15.764729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fad33c8add3e499caa13e45fc7af5a4f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:15.766629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:15.888923 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fad33c8add3e499caa13e45fc7af5a4f', 'request_id': 'req-693cd3ae-d32f-4776-86f6-821c787a0a51'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:15.957013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fad33c8add3e499caa13e45fc7af5a4f', 'request_id': 'req-693cd3ae-d32f-4776-86f6-821c787a0a51'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:15.963762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fad33c8add3e499caa13e45fc7af5a4f', 'request_id': 'req-693cd3ae-d32f-4776-86f6-821c787a0a51'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:15.974857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] The token cache is being invalidate because project fad33c8add3e499caa13e45fc7af5a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:36:15.975083 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:15.975705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:15.977204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1160/2304] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:36:15 2022] DELETE /identity/v3/projects/fad33c8add3e499caa13e45fc7af5a4f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:36:17.180765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c2044-62e2-4ba1-bcd9-a000ea0cf71c service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:17.189179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c2044-62e2-4ba1-bcd9-a000ea0cf71c service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:17.208036 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:17.236925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:17.237502 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:17.238314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:17.238580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:17.240306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:17.304319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:17.305727 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:17.315242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1145/2305] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:20.745610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:20.748122 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:20.833394 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cd3ae-d32f-4776-86f6-821c787a0a51 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:20.838321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:20.841737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:20.841737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:20.841737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:20.841737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:20.853867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:20.855483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:20.866631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1161/2306] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:20.880507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:20.883582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:20.908072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533071ae-30f4-4a8c-8dd8-679c61d5d54d service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:20.915936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:20.917072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:20.917614 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:20.918163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] PATH_INFO: `/v3/users/e0dd5e32e4a642c8ac9e44310e0a6a5b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:20.934287 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0dd5e32e4a642c8ac9e44310e0a6a5b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:20.936306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:21.114038 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0dd5e32e4a642c8ac9e44310e0a6a5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.121378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0dd5e32e4a642c8ac9e44310e0a6a5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.135574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0dd5e32e4a642c8ac9e44310e0a6a5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.214100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0dd5e32e4a642c8ac9e44310e0a6a5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.214709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-212e2ebd-68d0-46a9-8176-9bfe2d31b8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0dd5e32e4a642c8ac9e44310e0a6a5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.221045 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1146/2307] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:36:20 2022] DELETE /identity/v3/users/e0dd5e32e4a642c8ac9e44310e0a6a5b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:36:21.467360 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:21.480124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf585558-21ca-4b2e-89a6-9ea1f524acb4 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:21.490062 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:21.538880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:21.539507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:21.539773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:21.540041 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] PATH_INFO: `/v3/projects/e3faf7ad1fea4ab8b08c98a10491ec11` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:21.550259 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3faf7ad1fea4ab8b08c98a10491ec11)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:21.552716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:21.637996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3faf7ad1fea4ab8b08c98a10491ec11', 'request_id': 'req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.711405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3faf7ad1fea4ab8b08c98a10491ec11', 'request_id': 'req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.718480 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3faf7ad1fea4ab8b08c98a10491ec11', 'request_id': 'req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.728977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] The token cache is being invalidate because project e3faf7ad1fea4ab8b08c98a10491ec11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:36:21.729259 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.729875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ca74a138-d5a2-429f-94f8-10d7e9fd1f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:36:21.731366 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1162/2308] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:36:21 2022] DELETE /identity/v3/projects/e3faf7ad1fea4ab8b08c98a10491ec11 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:36:21.736527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b553db-ab3a-4bb4-846d-06e34f732766 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:21.736806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b553db-ab3a-4bb4-846d-06e34f732766 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:21.737074 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b553db-ab3a-4bb4-846d-06e34f732766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:21.757174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-95b553db-ab3a-4bb4-846d-06e34f732766 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:21.758344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-95b553db-ab3a-4bb4-846d-06e34f732766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:21.787030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:21.816816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1147/2309] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:21.825043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1ed7-8d06-49de-8d3d-c599fa1b69ff None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:21.825535 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1ed7-8d06-49de-8d3d-c599fa1b69ff None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:21.825654 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1ed7-8d06-49de-8d3d-c599fa1b69ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:21.844578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c81e1ed7-8d06-49de-8d3d-c599fa1b69ff None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:21.845791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c81e1ed7-8d06-49de-8d3d-c599fa1b69ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:21.854142 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1163/2310] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:21.858267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b553db-ab3a-4bb4-846d-06e34f732766 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:21.868224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b553db-ab3a-4bb4-846d-06e34f732766 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:21.872689 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:21.873275 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:21.873554 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:21.873822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:21.874453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:36:21.874752 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:21.876352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:21.888201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1148/2311] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:36:21.892326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e1ed7-8d06-49de-8d3d-c599fa1b69ff None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:21.901740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e1ed7-8d06-49de-8d3d-c599fa1b69ff None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:21.906319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ec306-d1c5-4abb-b26e-72485a74ae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:21.906911 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ec306-d1c5-4abb-b26e-72485a74ae22 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:21.907184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ec306-d1c5-4abb-b26e-72485a74ae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:21.907450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ec306-d1c5-4abb-b26e-72485a74ae22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:21.908073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ec306-d1c5-4abb-b26e-72485a74ae22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:21.910552 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ec306-d1c5-4abb-b26e-72485a74ae22 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:21.980727 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1164/2312] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:36:21 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:36:21.984739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:21.995112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dec5d-c357-42ae-bef9-19d837c4bc98 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:22.050103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b533a7-8e3b-4d97-a071-258cb3edb452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:22.051310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b533a7-8e3b-4d97-a071-258cb3edb452 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:22.051588 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b533a7-8e3b-4d97-a071-258cb3edb452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:22.051859 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b533a7-8e3b-4d97-a071-258cb3edb452 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:22.052509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b533a7-8e3b-4d97-a071-258cb3edb452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:22.054403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b533a7-8e3b-4d97-a071-258cb3edb452 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:22.151628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1149/2313] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:36:21 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:36:22.156070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ec306-d1c5-4abb-b26e-72485a74ae22 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:22.166644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ec306-d1c5-4abb-b26e-72485a74ae22 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:22.171212 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67106e2-e6aa-418e-82ef-d462fc3741a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:22.171790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67106e2-e6aa-418e-82ef-d462fc3741a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:22.172065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67106e2-e6aa-418e-82ef-d462fc3741a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:22.172375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67106e2-e6aa-418e-82ef-d462fc3741a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:22.173046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67106e2-e6aa-418e-82ef-d462fc3741a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:22.174621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67106e2-e6aa-418e-82ef-d462fc3741a8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:22.185490 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1165/2314] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:36:22.189439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b533a7-8e3b-4d97-a071-258cb3edb452 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:22.199506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b533a7-8e3b-4d97-a071-258cb3edb452 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:22.204681 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced618e-e092-41c5-8b43-d6d3b2f60af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:22.205250 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced618e-e092-41c5-8b43-d6d3b2f60af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:22.205542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced618e-e092-41c5-8b43-d6d3b2f60af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:22.205813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced618e-e092-41c5-8b43-d6d3b2f60af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:22.206944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced618e-e092-41c5-8b43-d6d3b2f60af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:22.208581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced618e-e092-41c5-8b43-d6d3b2f60af6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:22.219027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1150/2315] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:36:22.223742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67106e2-e6aa-418e-82ef-d462fc3741a8 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:22.233986 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67106e2-e6aa-418e-82ef-d462fc3741a8 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:22.238668 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:22.239330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:22.239612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:22.239887 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] PATH_INFO: `/v3/projects/0a7896472f034eb4b3b028d528cfbac0/users/54fcced6d8d943608b940d1dab001540/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:22.254315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7896472f034eb4b3b028d528cfbac0, user_id=54fcced6d8d943608b940d1dab001540, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:22.259359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:22.356652 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1166/2316] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:36:22 2022] PUT /identity/v3/projects/0a7896472f034eb4b3b028d528cfbac0/users/54fcced6d8d943608b940d1dab001540/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:36:22.361801 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81fc5b-d9ac-40dc-9cc2-0c25dbc1eebc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:22.362191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81fc5b-d9ac-40dc-9cc2-0c25dbc1eebc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:22.362496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81fc5b-d9ac-40dc-9cc2-0c25dbc1eebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:22.381212 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1d81fc5b-d9ac-40dc-9cc2-0c25dbc1eebc None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:22.382388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1d81fc5b-d9ac-40dc-9cc2-0c25dbc1eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:22.438594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1151/2317] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:22.448113 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:22.450076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:22.484044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:22.505797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfbf2b-3870-4f66-88c0-00656c41aa1c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:22.509499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:22.510102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:22.510574 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:22.510954 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:22.520808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:22.522225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:22.531457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1167/2318] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:27.704236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cae575-068b-43bf-983f-5f98f430e99f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:27.704819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cae575-068b-43bf-983f-5f98f430e99f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:27.704922 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cae575-068b-43bf-983f-5f98f430e99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:27.734665 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a0cae575-068b-43bf-983f-5f98f430e99f None None] MFA Rules not processed for user `54fcced6d8d943608b940d1dab001540`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:27.735873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0cae575-068b-43bf-983f-5f98f430e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:27.767119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:27.806131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1152/2319] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:36:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:36:27.823518 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:27.825309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:27.904004 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e220d6-e67b-4d4a-8945-9001e53e1fc2 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:27.910126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b865a3e-f985-4259-b315-6b6d6b941ff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:27.910792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865a3e-f985-4259-b315-6b6d6b941ff0 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:27.911000 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865a3e-f985-4259-b315-6b6d6b941ff0 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:27.911265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865a3e-f985-4259-b315-6b6d6b941ff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:27.919689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b865a3e-f985-4259-b315-6b6d6b941ff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:27.921320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b865a3e-f985-4259-b315-6b6d6b941ff0 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:27.932343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1168/2320] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:29.669829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cae575-068b-43bf-983f-5f98f430e99f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:29.671357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0cae575-068b-43bf-983f-5f98f430e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:29.685480 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cae575-068b-43bf-983f-5f98f430e99f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:29.689897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:29.690746 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:29.691529 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:29.691970 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:29.693993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:29.704965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:29.708130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:29.717884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1153/2321] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:30.108442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd083-d2a1-4284-bf8d-5a311a6af3eb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:30.108442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd083-d2a1-4284-bf8d-5a311a6af3eb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:30.108442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacd083-d2a1-4284-bf8d-5a311a6af3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:30.125205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-fcacd083-d2a1-4284-bf8d-5a311a6af3eb None None] MFA Rules not processed for user `54fcced6d8d943608b940d1dab001540`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:36:30.126526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fcacd083-d2a1-4284-bf8d-5a311a6af3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:30.139608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1169/2322] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:36:30.722804 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:30.736504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d5577-d90c-4357-8a96-2fea33fccf7b service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:30.743583 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:30.744189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:30.744506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:30.744769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:30.746794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:30.757522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:30.759010 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:30.767895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1154/2323] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:31.246758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacd083-d2a1-4284-bf8d-5a311a6af3eb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:31.248462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fcacd083-d2a1-4284-bf8d-5a311a6af3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:31.369251 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacd083-d2a1-4284-bf8d-5a311a6af3eb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:31.374423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:31.375039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:31.375318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:31.375596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:31.377448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:31.389909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:31.391532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:31.403309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1170/2324] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:35.246747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:35.261515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e9604-a0c5-4b01-af7c-d5fd9a9d6c6b service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:35.267241 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:35.267851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:35.268130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:35.268437 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:35.270183 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:35.282908 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:35.286521 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:35.298057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1155/2325] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:40.742191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:36:40.754805 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1307ba-e713-4a8c-a210-0aa8b1ee4b75 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:36:40.760258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:36:40.760878 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:36:40.761158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:36:40.761428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:36:40.763149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:36:40.775877 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:36:40.780499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:36:40.787776 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1171/2326] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:36:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:36:45.293435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=32 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:36:50.783457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=87 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:02.508439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:02.517699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bc0c4-ce6f-4cf2-b1ea-056c03fe7584 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:02.521289 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:02.521872 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:02.522151 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:02.522422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:02.524118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:02.536220 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:02.537646 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:02.547178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1156/2327] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:05.883858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:05.885457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:05.900078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b3f9b-fc92-46d0-af9c-72678ade2f5f service nova] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:05.904843 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:05.905464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:05.905745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:05.906018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] PATH_INFO: `/v3/users/7c096064e6ec443a9e8755bc2e68dbb7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:05.907160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c096064e6ec443a9e8755bc2e68dbb7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:05.909096 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:06.085080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c096064e6ec443a9e8755bc2e68dbb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.085680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c096064e6ec443a9e8755bc2e68dbb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.098413 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c096064e6ec443a9e8755bc2e68dbb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.105205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c096064e6ec443a9e8755bc2e68dbb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.105873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-a7864ab7-e218-4b4f-bf1a-744002892289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c096064e6ec443a9e8755bc2e68dbb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.173484 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1172/2328] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:05 2022] DELETE /identity/v3/users/7c096064e6ec443a9e8755bc2e68dbb7 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:06.399000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:06.411286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9e367-a49d-458a-a469-f27c59c2c56f service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:06.471370 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:06.471993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:06.472924 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:06.473149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] PATH_INFO: `/v3/projects/7fa2196e343c45b5b741c39417ba9ef8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:06.485841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fa2196e343c45b5b741c39417ba9ef8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:06.485841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:06.573892 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fa2196e343c45b5b741c39417ba9ef8', 'request_id': 'req-8621596b-6ca6-43f5-bae6-1833e37cae45'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.579654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fa2196e343c45b5b741c39417ba9ef8', 'request_id': 'req-8621596b-6ca6-43f5-bae6-1833e37cae45'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.586444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fa2196e343c45b5b741c39417ba9ef8', 'request_id': 'req-8621596b-6ca6-43f5-bae6-1833e37cae45'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.652825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] The token cache is being invalidate because project 7fa2196e343c45b5b741c39417ba9ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:06.653113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.653722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8621596b-6ca6-43f5-bae6-1833e37cae45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:06.655179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1157/2329] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:06 2022] DELETE /identity/v3/projects/7fa2196e343c45b5b741c39417ba9ef8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:37:06.661019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02f91-a48d-4f54-8d06-528b6a1f413b None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:06.661309 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02f91-a48d-4f54-8d06-528b6a1f413b None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:06.661582 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a02f91-a48d-4f54-8d06-528b6a1f413b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:06.680926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-69a02f91-a48d-4f54-8d06-528b6a1f413b None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:06.682119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-69a02f91-a48d-4f54-8d06-528b6a1f413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:06.745748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1173/2330] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:06.755232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f80c5-233c-4573-8271-dbd7bfe73df6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:06.755754 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f80c5-233c-4573-8271-dbd7bfe73df6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:06.756328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f80c5-233c-4573-8271-dbd7bfe73df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:06.779798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8a8f80c5-233c-4573-8271-dbd7bfe73df6 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:06.781075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8a8f80c5-233c-4573-8271-dbd7bfe73df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:06.789513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1158/2331] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:06.793705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a02f91-a48d-4f54-8d06-528b6a1f413b None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:06.803493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a02f91-a48d-4f54-8d06-528b6a1f413b None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:06.808024 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:06.808713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:06.808990 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:06.809260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:06.809891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:37:06.810196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:06.811776 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:06.823979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1174/2332] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:37:06.828463 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f80c5-233c-4573-8271-dbd7bfe73df6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:06.838405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f80c5-233c-4573-8271-dbd7bfe73df6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:06.843301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fec238-23b8-4294-a8ce-0bd949f8b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:06.843969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fec238-23b8-4294-a8ce-0bd949f8b9fd admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:06.844245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fec238-23b8-4294-a8ce-0bd949f8b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:06.844543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fec238-23b8-4294-a8ce-0bd949f8b9fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:06.845162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fec238-23b8-4294-a8ce-0bd949f8b9fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:06.847029 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fec238-23b8-4294-a8ce-0bd949f8b9fd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:06.928528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1159/2333] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:06 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:06.934625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:06.945103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657299c-94da-44a4-a872-b609ef44a878 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:07.000288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154ebbe-f5f1-44c7-9296-de5d6afe5ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:07.000902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154ebbe-f5f1-44c7-9296-de5d6afe5ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:07.001183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154ebbe-f5f1-44c7-9296-de5d6afe5ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:07.001455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154ebbe-f5f1-44c7-9296-de5d6afe5ccc admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:07.002103 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154ebbe-f5f1-44c7-9296-de5d6afe5ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:07.004553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154ebbe-f5f1-44c7-9296-de5d6afe5ccc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:07.012354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:07.076284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1175/2334] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:06 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:07.080352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fec238-23b8-4294-a8ce-0bd949f8b9fd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:07.091743 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fec238-23b8-4294-a8ce-0bd949f8b9fd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:07.096580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbb61f-5c18-4698-a061-c003ea8a55b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:07.098152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbb61f-5c18-4698-a061-c003ea8a55b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:07.098152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbb61f-5c18-4698-a061-c003ea8a55b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:07.098152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbb61f-5c18-4698-a061-c003ea8a55b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:07.098444 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbb61f-5c18-4698-a061-c003ea8a55b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:07.099975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbb61f-5c18-4698-a061-c003ea8a55b2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:07.110430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1160/2335] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:37:07.114387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154ebbe-f5f1-44c7-9296-de5d6afe5ccc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:07.124262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154ebbe-f5f1-44c7-9296-de5d6afe5ccc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:07.129465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f43c8-68bd-42fc-88ae-4b5ee6a57cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:07.130050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f43c8-68bd-42fc-88ae-4b5ee6a57cab admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:07.130331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f43c8-68bd-42fc-88ae-4b5ee6a57cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:07.130608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f43c8-68bd-42fc-88ae-4b5ee6a57cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:07.131240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f43c8-68bd-42fc-88ae-4b5ee6a57cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:07.132858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f43c8-68bd-42fc-88ae-4b5ee6a57cab admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:07.143386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1176/2336] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:07.147923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbb61f-5c18-4698-a061-c003ea8a55b2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:07.157742 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbb61f-5c18-4698-a061-c003ea8a55b2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:07.162353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:07.162976 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:07.163249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:07.163515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] PATH_INFO: `/v3/projects/5e4106fa08064c399dc3fb97c51aeba1/users/8486cb52fc76461ea74161fa72d0fa3b/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:07.178204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4106fa08064c399dc3fb97c51aeba1, user_id=8486cb52fc76461ea74161fa72d0fa3b, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:07.180944 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:07.242181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1161/2337] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:07 2022] PUT /identity/v3/projects/5e4106fa08064c399dc3fb97c51aeba1/users/8486cb52fc76461ea74161fa72d0fa3b/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:07.260795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41695f8d-1fed-4921-8a59-ec6744981221 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:07.261082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41695f8d-1fed-4921-8a59-ec6744981221 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:07.261352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41695f8d-1fed-4921-8a59-ec6744981221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:07.288948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-41695f8d-1fed-4921-8a59-ec6744981221 None None] MFA Rules not processed for user `8486cb52fc76461ea74161fa72d0fa3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:07.290141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-41695f8d-1fed-4921-8a59-ec6744981221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:07.354179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1177/2338] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:07.366153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:07.367813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:07.382798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:07.442751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a11d21-1c0d-48e1-b93f-bfaffaf36fdf admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:07.448248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706c452-47db-4857-8159-d0072bbc5794 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:07.451195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706c452-47db-4857-8159-d0072bbc5794 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:07.451195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706c452-47db-4857-8159-d0072bbc5794 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:07.451195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706c452-47db-4857-8159-d0072bbc5794 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:07.457391 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706c452-47db-4857-8159-d0072bbc5794 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:07.458972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706c452-47db-4857-8159-d0072bbc5794 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:07.467590 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1162/2339] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:10.605321 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41695f8d-1fed-4921-8a59-ec6744981221 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:10.663835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41695f8d-1fed-4921-8a59-ec6744981221 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:10.668289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:10.668916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:10.669196 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:10.669467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] PATH_INFO: `/v3/users/8486cb52fc76461ea74161fa72d0fa3b` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:10.670561 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8486cb52fc76461ea74161fa72d0fa3b)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:10.672492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:10.896387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8486cb52fc76461ea74161fa72d0fa3b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:10.897038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8486cb52fc76461ea74161fa72d0fa3b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:10.909302 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8486cb52fc76461ea74161fa72d0fa3b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:10.914531 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:10.917292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8486cb52fc76461ea74161fa72d0fa3b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:10.917970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8486cb52fc76461ea74161fa72d0fa3b'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:10.979311 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1178/2340] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:10 2022] DELETE /identity/v3/users/8486cb52fc76461ea74161fa72d0fa3b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:10.983788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33e466-99b6-4f0e-96bb-fef22b5e32c5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:10.984341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33e466-99b6-4f0e-96bb-fef22b5e32c5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:10.984341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33e466-99b6-4f0e-96bb-fef22b5e32c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:11.006344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1c33e466-99b6-4f0e-96bb-fef22b5e32c5 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:11.007546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1c33e466-99b6-4f0e-96bb-fef22b5e32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:11.069116 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1163/2341] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:11.080306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:11.084903 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:11.159384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a6462-a84c-4519-9053-e77fcb79df0f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:11.163411 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f8556-5f7a-4929-bd62-d2edee7c2030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:11.164287 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f8556-5f7a-4929-bd62-d2edee7c2030 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:11.164796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f8556-5f7a-4929-bd62-d2edee7c2030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:11.165053 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f8556-5f7a-4929-bd62-d2edee7c2030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:11.180534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f8556-5f7a-4929-bd62-d2edee7c2030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:11.182316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f8556-5f7a-4929-bd62-d2edee7c2030 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:11.196236 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1179/2342] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:11.624659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33e466-99b6-4f0e-96bb-fef22b5e32c5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:11.626175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1c33e466-99b6-4f0e-96bb-fef22b5e32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:11.640283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33e466-99b6-4f0e-96bb-fef22b5e32c5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:11.645562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:11.654113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:11.654113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:11.654113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] PATH_INFO: `/v3/projects/5e4106fa08064c399dc3fb97c51aeba1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:11.654113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4106fa08064c399dc3fb97c51aeba1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:11.654113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:11.774745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4106fa08064c399dc3fb97c51aeba1', 'request_id': 'req-620cbb1f-685b-4700-9903-5136c1c97da2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:11.780340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4106fa08064c399dc3fb97c51aeba1', 'request_id': 'req-620cbb1f-685b-4700-9903-5136c1c97da2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:11.787246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4106fa08064c399dc3fb97c51aeba1', 'request_id': 'req-620cbb1f-685b-4700-9903-5136c1c97da2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:11.853243 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] The token cache is being invalidate because project 5e4106fa08064c399dc3fb97c51aeba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:11.853528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:11.854149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-620cbb1f-685b-4700-9903-5136c1c97da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:11.855608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1164/2343] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:11 2022] DELETE /identity/v3/projects/5e4106fa08064c399dc3fb97c51aeba1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:11.860532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd6c5-450f-483c-aa45-bfb0a5b71cb1 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:11.864130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd6c5-450f-483c-aa45-bfb0a5b71cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:11.864130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd6c5-450f-483c-aa45-bfb0a5b71cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:11.880548 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-07fdd6c5-450f-483c-aa45-bfb0a5b71cb1 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:11.881701 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-07fdd6c5-450f-483c-aa45-bfb0a5b71cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:11.938959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1180/2344] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:11.946676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6d20-4ad2-4a82-b1d7-6356c59bb123 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:11.947049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6d20-4ad2-4a82-b1d7-6356c59bb123 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:11.947398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6d20-4ad2-4a82-b1d7-6356c59bb123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:11.967761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5bdc6d20-4ad2-4a82-b1d7-6356c59bb123 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:11.968983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5bdc6d20-4ad2-4a82-b1d7-6356c59bb123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:11.977555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1165/2345] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:11.981812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd6c5-450f-483c-aa45-bfb0a5b71cb1 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:11.991134 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd6c5-450f-483c-aa45-bfb0a5b71cb1 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:11.995583 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:11.996157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:11.996469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:11.996736 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:11.997359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:37:11.997655 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:11.999211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.011072 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1181/2346] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:37:12.015112 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc6d20-4ad2-4a82-b1d7-6356c59bb123 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.024806 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc6d20-4ad2-4a82-b1d7-6356c59bb123 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.029419 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb526e3-3e9c-4c77-9100-b5178bade22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.030003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb526e3-3e9c-4c77-9100-b5178bade22d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.030281 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb526e3-3e9c-4c77-9100-b5178bade22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.030581 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb526e3-3e9c-4c77-9100-b5178bade22d admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.031213 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb526e3-3e9c-4c77-9100-b5178bade22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.033152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb526e3-3e9c-4c77-9100-b5178bade22d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.047981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:12.112389 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1166/2347] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:12 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:12.116974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.126825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf1a4d-5aad-46b4-be1b-524f2027ffc4 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.182336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68112e8-681d-460b-9233-4f632fa5c0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.182926 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68112e8-681d-460b-9233-4f632fa5c0ba admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.183202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68112e8-681d-460b-9233-4f632fa5c0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.183472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68112e8-681d-460b-9233-4f632fa5c0ba admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.184082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68112e8-681d-460b-9233-4f632fa5c0ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.186032 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68112e8-681d-460b-9233-4f632fa5c0ba admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.265034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1182/2348] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:12 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:12.269853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb526e3-3e9c-4c77-9100-b5178bade22d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.280092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb526e3-3e9c-4c77-9100-b5178bade22d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.284798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f80446-6fae-4730-9335-8c408eb1e9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.285388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f80446-6fae-4730-9335-8c408eb1e9fa admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.285662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f80446-6fae-4730-9335-8c408eb1e9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.285940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f80446-6fae-4730-9335-8c408eb1e9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.286571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f80446-6fae-4730-9335-8c408eb1e9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.288158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f80446-6fae-4730-9335-8c408eb1e9fa admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.298198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1167/2349] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:12.302481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68112e8-681d-460b-9233-4f632fa5c0ba admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.313991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68112e8-681d-460b-9233-4f632fa5c0ba admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.320222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fcd8f-69b1-4a93-9b8a-2ab87685d9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.320861 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcd8f-69b1-4a93-9b8a-2ab87685d9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.321140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcd8f-69b1-4a93-9b8a-2ab87685d9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.321410 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcd8f-69b1-4a93-9b8a-2ab87685d9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.322044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fcd8f-69b1-4a93-9b8a-2ab87685d9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.323760 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fcd8f-69b1-4a93-9b8a-2ab87685d9d8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.325566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:12.336914 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1183/2350] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:37:12.341113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f80446-6fae-4730-9335-8c408eb1e9fa admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.351031 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f80446-6fae-4730-9335-8c408eb1e9fa admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.359714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b77cd-c5f3-41b0-ab81-54301ff091e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.360448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b77cd-c5f3-41b0-ab81-54301ff091e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.360737 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b77cd-c5f3-41b0-ab81-54301ff091e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.361014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b77cd-c5f3-41b0-ab81-54301ff091e7 admin admin] PATH_INFO: `/v3/projects/087d2d3475f64d5c949852d054638047/users/6e28a370599f4d2e96b3da8a2b66147d/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.377037 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b77cd-c5f3-41b0-ab81-54301ff091e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087d2d3475f64d5c949852d054638047, user_id=6e28a370599f4d2e96b3da8a2b66147d, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.380215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b77cd-c5f3-41b0-ab81-54301ff091e7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.443636 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1168/2351] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:12 2022] PUT /identity/v3/projects/087d2d3475f64d5c949852d054638047/users/6e28a370599f4d2e96b3da8a2b66147d/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:12.456491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b756ff1-becc-45f4-9b9c-b528b9c4e943 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.457044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b756ff1-becc-45f4-9b9c-b528b9c4e943 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.457391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b756ff1-becc-45f4-9b9c-b528b9c4e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.488452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-2b756ff1-becc-45f4-9b9c-b528b9c4e943 None None] MFA Rules not processed for user `6e28a370599f4d2e96b3da8a2b66147d`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:12.489723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2b756ff1-becc-45f4-9b9c-b528b9c4e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:12.555974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1184/2352] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:12.560490 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b77cd-c5f3-41b0-ab81-54301ff091e7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.623563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b77cd-c5f3-41b0-ab81-54301ff091e7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.628254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc58b81-da59-49b5-97f8-ad776cce404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.629433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58b81-da59-49b5-97f8-ad776cce404c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.629733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58b81-da59-49b5-97f8-ad776cce404c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.630030 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58b81-da59-49b5-97f8-ad776cce404c admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.630671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc58b81-da59-49b5-97f8-ad776cce404c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.632626 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc58b81-da59-49b5-97f8-ad776cce404c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.707951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1169/2353] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:12 2022] POST /identity/v3/projects => generated 363 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:12.711006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b756ff1-becc-45f4-9b9c-b528b9c4e943 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.722107 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b756ff1-becc-45f4-9b9c-b528b9c4e943 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.777093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b16b9-c291-4bc1-b27d-317b5dd62ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.777690 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b16b9-c291-4bc1-b27d-317b5dd62ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.777967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b16b9-c291-4bc1-b27d-317b5dd62ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.778234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b16b9-c291-4bc1-b27d-317b5dd62ee2 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.778864 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b16b9-c291-4bc1-b27d-317b5dd62ee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.780796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b16b9-c291-4bc1-b27d-317b5dd62ee2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.876297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1185/2354] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:12 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:12.880132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc58b81-da59-49b5-97f8-ad776cce404c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.890249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc58b81-da59-49b5-97f8-ad776cce404c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.894827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcceb98-9efd-4aab-92b5-599a74a1db9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.895407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcceb98-9efd-4aab-92b5-599a74a1db9c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.895682 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcceb98-9efd-4aab-92b5-599a74a1db9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.895951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcceb98-9efd-4aab-92b5-599a74a1db9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.896609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcceb98-9efd-4aab-92b5-599a74a1db9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.898180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcceb98-9efd-4aab-92b5-599a74a1db9c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:12.908718 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1170/2355] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:12.913416 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b16b9-c291-4bc1-b27d-317b5dd62ee2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:12.923407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b16b9-c291-4bc1-b27d-317b5dd62ee2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:12.928669 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e6824-3c94-46de-a28f-e224023d3db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:12.929312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e6824-3c94-46de-a28f-e224023d3db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:12.929588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e6824-3c94-46de-a28f-e224023d3db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:12.929862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e6824-3c94-46de-a28f-e224023d3db3 admin admin] PATH_INFO: `/v3/projects/5b29f73eccaa4e688563b1ea99f2a4ec/users/9f84edef49bc44c1a75445890ef566b6/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:12.945460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e6824-3c94-46de-a28f-e224023d3db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b29f73eccaa4e688563b1ea99f2a4ec, user_id=9f84edef49bc44c1a75445890ef566b6, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:12.948390 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e6824-3c94-46de-a28f-e224023d3db3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:13.031141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1186/2356] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:12 2022] PUT /identity/v3/projects/5b29f73eccaa4e688563b1ea99f2a4ec/users/9f84edef49bc44c1a75445890ef566b6/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:37:13.035687 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcceb98-9efd-4aab-92b5-599a74a1db9c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:13.045943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcceb98-9efd-4aab-92b5-599a74a1db9c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:13.105142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10554cd6-6ae2-4077-b953-89bccf0c815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:13.105763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554cd6-6ae2-4077-b953-89bccf0c815d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:13.106041 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554cd6-6ae2-4077-b953-89bccf0c815d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:13.106310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554cd6-6ae2-4077-b953-89bccf0c815d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:13.106940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10554cd6-6ae2-4077-b953-89bccf0c815d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:13.108553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10554cd6-6ae2-4077-b953-89bccf0c815d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:13.118677 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1171/2357] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:13.122918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e6824-3c94-46de-a28f-e224023d3db3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:13.133294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e6824-3c94-46de-a28f-e224023d3db3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:13.137867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:13.138528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:13.138802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:13.139070 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] PATH_INFO: `/v3/projects/5b29f73eccaa4e688563b1ea99f2a4ec/users/9f84edef49bc44c1a75445890ef566b6/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:13.140979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b29f73eccaa4e688563b1ea99f2a4ec, user_id=9f84edef49bc44c1a75445890ef566b6, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:13.143588 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:13.154160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1187/2358] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:13 2022] PUT /identity/v3/projects/5b29f73eccaa4e688563b1ea99f2a4ec/users/9f84edef49bc44c1a75445890ef566b6/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:13.164896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59661576-56df-44cd-81e8-e0aa3d57ee92 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:13.165182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59661576-56df-44cd-81e8-e0aa3d57ee92 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:13.165454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59661576-56df-44cd-81e8-e0aa3d57ee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:13.192957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-59661576-56df-44cd-81e8-e0aa3d57ee92 None None] MFA Rules not processed for user `9f84edef49bc44c1a75445890ef566b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:13.194153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-59661576-56df-44cd-81e8-e0aa3d57ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:13.211175 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:13.261798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1172/2359] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:13.273762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:13.275392 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:13.316869 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:13.331994 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26e53b-4c18-4cf6-99df-9dd0381b05b3 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:13.335623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb349b9f-e7b5-4386-b4b1-bb48f6fe02fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:13.336208 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb349b9f-e7b5-4386-b4b1-bb48f6fe02fa service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:13.336514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb349b9f-e7b5-4386-b4b1-bb48f6fe02fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:13.336786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb349b9f-e7b5-4386-b4b1-bb48f6fe02fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:13.345997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb349b9f-e7b5-4386-b4b1-bb48f6fe02fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:13.348067 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb349b9f-e7b5-4386-b4b1-bb48f6fe02fa service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:13.357273 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1188/2360] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:13.563791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59661576-56df-44cd-81e8-e0aa3d57ee92 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:13.574040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59661576-56df-44cd-81e8-e0aa3d57ee92 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:13.577769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bacb91-dfba-46fe-a6f6-012d7ff6f2ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:13.578359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bacb91-dfba-46fe-a6f6-012d7ff6f2ae service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:13.578637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bacb91-dfba-46fe-a6f6-012d7ff6f2ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:13.578942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bacb91-dfba-46fe-a6f6-012d7ff6f2ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:13.637678 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bacb91-dfba-46fe-a6f6-012d7ff6f2ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:13.639106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bacb91-dfba-46fe-a6f6-012d7ff6f2ae service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:13.648503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1173/2361] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:23.361702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=9 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:23.643798 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=49 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:26.255755 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb349b9f-e7b5-4386-b4b1-bb48f6fe02fa service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:26.314967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb349b9f-e7b5-4386-b4b1-bb48f6fe02fa service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:26.319874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:26.320516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:26.320793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:26.321061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] PATH_INFO: `/v3/users/6e28a370599f4d2e96b3da8a2b66147d` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:26.322206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e28a370599f4d2e96b3da8a2b66147d)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:26.324369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:26.449340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e28a370599f4d2e96b3da8a2b66147d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:26.450130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e28a370599f4d2e96b3da8a2b66147d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:26.462555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e28a370599f4d2e96b3da8a2b66147d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:26.469262 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e28a370599f4d2e96b3da8a2b66147d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:26.469880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e28a370599f4d2e96b3da8a2b66147d'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:26.555260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1189/2362] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:26 2022] DELETE /identity/v3/users/6e28a370599f4d2e96b3da8a2b66147d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:26.559702 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b4f9c-e7ba-4c6f-a258-0d18a3ec6645 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:26.564344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b4f9c-e7ba-4c6f-a258-0d18a3ec6645 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:26.564344 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b4f9c-e7ba-4c6f-a258-0d18a3ec6645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:26.579690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-555b4f9c-e7ba-4c6f-a258-0d18a3ec6645 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:26.580935 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-555b4f9c-e7ba-4c6f-a258-0d18a3ec6645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:26.639049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1174/2363] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:26.648632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:26.650038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:26.705534 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c59812-7706-4df7-891c-79c4de7d89b7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:26.709084 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d259-668f-4f35-aeea-5cb52f11af73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:26.709670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d259-668f-4f35-aeea-5cb52f11af73 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:26.709942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d259-668f-4f35-aeea-5cb52f11af73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:26.710205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d259-668f-4f35-aeea-5cb52f11af73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:26.718761 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180d259-668f-4f35-aeea-5cb52f11af73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:26.720159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180d259-668f-4f35-aeea-5cb52f11af73 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:26.729260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1190/2364] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:27.089395 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b4f9c-e7ba-4c6f-a258-0d18a3ec6645 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:27.091017 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-555b4f9c-e7ba-4c6f-a258-0d18a3ec6645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:27.104016 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b4f9c-e7ba-4c6f-a258-0d18a3ec6645 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:27.109412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:27.110055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:27.110358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:27.110629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] PATH_INFO: `/v3/projects/087d2d3475f64d5c949852d054638047` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:27.111758 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087d2d3475f64d5c949852d054638047)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:27.113658 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:27.204553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087d2d3475f64d5c949852d054638047', 'request_id': 'req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.210094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087d2d3475f64d5c949852d054638047', 'request_id': 'req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.216520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087d2d3475f64d5c949852d054638047', 'request_id': 'req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.309346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] The token cache is being invalidate because project 087d2d3475f64d5c949852d054638047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:27.309632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.310314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.311789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1175/2365] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:27 2022] DELETE /identity/v3/projects/087d2d3475f64d5c949852d054638047 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:27.315742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d259-668f-4f35-aeea-5cb52f11af73 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:27.325743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d259-668f-4f35-aeea-5cb52f11af73 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:27.344051 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:27.380829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:27.381506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:27.381787 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:27.382060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] PATH_INFO: `/v3/users/9f84edef49bc44c1a75445890ef566b6` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:27.383217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f84edef49bc44c1a75445890ef566b6)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:27.385516 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:27.540147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f84edef49bc44c1a75445890ef566b6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.540808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f84edef49bc44c1a75445890ef566b6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.552501 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f84edef49bc44c1a75445890ef566b6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.559729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f84edef49bc44c1a75445890ef566b6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.562311 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-2f6fa687-34e0-43e3-af20-f6ccebcde3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f84edef49bc44c1a75445890ef566b6'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:27.624433 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1191/2366] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:27 2022] DELETE /identity/v3/users/9f84edef49bc44c1a75445890ef566b6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:27.986040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:27.987487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:28.000776 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6025bb9-66f7-4ed5-852c-6016aa1b4f41 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:28.057835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:28.058459 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.058740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.059014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] PATH_INFO: `/v3/projects/5b29f73eccaa4e688563b1ea99f2a4ec` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.060109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b29f73eccaa4e688563b1ea99f2a4ec)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:28.062037 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:28.146328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:28.162194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b29f73eccaa4e688563b1ea99f2a4ec', 'request_id': 'req-605f8674-b1e1-4f36-b72e-72e638b6269a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:28.167711 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b29f73eccaa4e688563b1ea99f2a4ec', 'request_id': 'req-605f8674-b1e1-4f36-b72e-72e638b6269a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:28.173959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b29f73eccaa4e688563b1ea99f2a4ec', 'request_id': 'req-605f8674-b1e1-4f36-b72e-72e638b6269a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:28.240739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] The token cache is being invalidate because project 5b29f73eccaa4e688563b1ea99f2a4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:28.241035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:28.242150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-605f8674-b1e1-4f36-b72e-72e638b6269a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:28.243608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1176/2367] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:27 2022] DELETE /identity/v3/projects/5b29f73eccaa4e688563b1ea99f2a4ec => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:28.248587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f4c04-4c14-45a7-8997-b2795875d7b3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.248874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f4c04-4c14-45a7-8997-b2795875d7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.249149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f4c04-4c14-45a7-8997-b2795875d7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.269446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-f68f4c04-4c14-45a7-8997-b2795875d7b3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:28.274677 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-f68f4c04-4c14-45a7-8997-b2795875d7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:28.329130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1192/2368] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:28.338543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943bb7-35d5-4ba5-a80b-53bd1efbf7b4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.338827 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943bb7-35d5-4ba5-a80b-53bd1efbf7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.339100 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0943bb7-35d5-4ba5-a80b-53bd1efbf7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.358165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c0943bb7-35d5-4ba5-a80b-53bd1efbf7b4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:28.359335 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c0943bb7-35d5-4ba5-a80b-53bd1efbf7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:28.367499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1177/2369] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:28.371642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f4c04-4c14-45a7-8997-b2795875d7b3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:28.381226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f4c04-4c14-45a7-8997-b2795875d7b3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:28.386221 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:28.386788 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.387069 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.387338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.387968 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:37:28.388291 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:28.389870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:28.401056 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1193/2370] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:37:28.405025 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0943bb7-35d5-4ba5-a80b-53bd1efbf7b4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:28.414475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0943bb7-35d5-4ba5-a80b-53bd1efbf7b4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:28.419028 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cab2-86a7-48e3-bb47-ae80fe518111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:28.419601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cab2-86a7-48e3-bb47-ae80fe518111 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.419876 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cab2-86a7-48e3-bb47-ae80fe518111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.420139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cab2-86a7-48e3-bb47-ae80fe518111 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.420775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9cab2-86a7-48e3-bb47-ae80fe518111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:28.422645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9cab2-86a7-48e3-bb47-ae80fe518111 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:28.488372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1178/2371] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:28 2022] POST /identity/v3/projects => generated 363 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:28.493117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:28.502858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3991-0577-4a20-a85a-db575728b02e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:28.527838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:28.558542 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe7f8e-beba-4d2f-9159-8b751140260b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:28.559131 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7f8e-beba-4d2f-9159-8b751140260b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.559406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7f8e-beba-4d2f-9159-8b751140260b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.559674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7f8e-beba-4d2f-9159-8b751140260b admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.560325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe7f8e-beba-4d2f-9159-8b751140260b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:28.562215 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe7f8e-beba-4d2f-9159-8b751140260b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:28.635885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1194/2372] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:28 2022] POST /identity/v3/users => generated 282 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:28.639932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cab2-86a7-48e3-bb47-ae80fe518111 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:28.650398 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cab2-86a7-48e3-bb47-ae80fe518111 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:28.655027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52036258-f0d7-4ddd-8fbf-adb0794fe29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:28.655562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52036258-f0d7-4ddd-8fbf-adb0794fe29c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.655841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52036258-f0d7-4ddd-8fbf-adb0794fe29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.656113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52036258-f0d7-4ddd-8fbf-adb0794fe29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.656779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52036258-f0d7-4ddd-8fbf-adb0794fe29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:28.658352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52036258-f0d7-4ddd-8fbf-adb0794fe29c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:28.668836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1179/2373] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:37:28.672599 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe7f8e-beba-4d2f-9159-8b751140260b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:28.682900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe7f8e-beba-4d2f-9159-8b751140260b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:28.687439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca404549-cfbd-4da6-b9a2-c4f85cd320c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:28.688017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca404549-cfbd-4da6-b9a2-c4f85cd320c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.688319 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca404549-cfbd-4da6-b9a2-c4f85cd320c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.688595 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca404549-cfbd-4da6-b9a2-c4f85cd320c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.689223 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca404549-cfbd-4da6-b9a2-c4f85cd320c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:28.690828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca404549-cfbd-4da6-b9a2-c4f85cd320c2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:28.700909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1195/2374] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:28.704954 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52036258-f0d7-4ddd-8fbf-adb0794fe29c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:28.714883 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52036258-f0d7-4ddd-8fbf-adb0794fe29c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:28.719564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37f3a1-68f5-4f29-9a4d-1d4dc090da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:28.720200 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f3a1-68f5-4f29-9a4d-1d4dc090da50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.720519 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f3a1-68f5-4f29-9a4d-1d4dc090da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.720816 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37f3a1-68f5-4f29-9a4d-1d4dc090da50 admin admin] PATH_INFO: `/v3/projects/3c95f93cb2414684936cb8df1da00738/users/7e1454b53c4844e7b6a9749cfc9cfd42/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.735662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37f3a1-68f5-4f29-9a4d-1d4dc090da50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c95f93cb2414684936cb8df1da00738, user_id=7e1454b53c4844e7b6a9749cfc9cfd42, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:28.738372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37f3a1-68f5-4f29-9a4d-1d4dc090da50 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:28.783900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1180/2375] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:28 2022] PUT /identity/v3/projects/3c95f93cb2414684936cb8df1da00738/users/7e1454b53c4844e7b6a9749cfc9cfd42/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:28.794678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac33b-8b60-4d6f-8651-2d8b6d54e610 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.794964 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac33b-8b60-4d6f-8651-2d8b6d54e610 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.795274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ac33b-8b60-4d6f-8651-2d8b6d54e610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.822700 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-756ac33b-8b60-4d6f-8651-2d8b6d54e610 None None] MFA Rules not processed for user `7e1454b53c4844e7b6a9749cfc9cfd42`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:28.823885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-756ac33b-8b60-4d6f-8651-2d8b6d54e610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:28.886855 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1196/2376] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:28.891314 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37f3a1-68f5-4f29-9a4d-1d4dc090da50 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:28.953177 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37f3a1-68f5-4f29-9a4d-1d4dc090da50 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:28.957707 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156c798-ae1b-492f-9602-660208d34e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:28.958299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156c798-ae1b-492f-9602-660208d34e9d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:28.958575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156c798-ae1b-492f-9602-660208d34e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:28.958842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156c798-ae1b-492f-9602-660208d34e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:28.959460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156c798-ae1b-492f-9602-660208d34e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:28.961409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156c798-ae1b-492f-9602-660208d34e9d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:29.065465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1181/2377] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:28 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:29.070325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac33b-8b60-4d6f-8651-2d8b6d54e610 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:29.081381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ac33b-8b60-4d6f-8651-2d8b6d54e610 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:29.137874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb89e0-418b-4b6a-b993-9c4886fa533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:29.138451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb89e0-418b-4b6a-b993-9c4886fa533b admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:29.138731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb89e0-418b-4b6a-b993-9c4886fa533b admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:29.139002 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb89e0-418b-4b6a-b993-9c4886fa533b admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:29.139623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb89e0-418b-4b6a-b993-9c4886fa533b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:29.141563 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb89e0-418b-4b6a-b993-9c4886fa533b admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:29.236409 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1197/2378] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:29 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:29.240312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156c798-ae1b-492f-9602-660208d34e9d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:29.251182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156c798-ae1b-492f-9602-660208d34e9d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:29.255799 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0287e54-b14b-4208-ba8e-77f75e0ee819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:29.256430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0287e54-b14b-4208-ba8e-77f75e0ee819 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:29.256708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0287e54-b14b-4208-ba8e-77f75e0ee819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:29.257007 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0287e54-b14b-4208-ba8e-77f75e0ee819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:29.257635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0287e54-b14b-4208-ba8e-77f75e0ee819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:29.259215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0287e54-b14b-4208-ba8e-77f75e0ee819 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:29.269597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1182/2379] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:29.273871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb89e0-418b-4b6a-b993-9c4886fa533b admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:29.283826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb89e0-418b-4b6a-b993-9c4886fa533b admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:29.294393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e6653-441c-4c15-bd7a-2bb5e96037bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:29.295055 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e6653-441c-4c15-bd7a-2bb5e96037bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:29.295336 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e6653-441c-4c15-bd7a-2bb5e96037bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:29.295611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e6653-441c-4c15-bd7a-2bb5e96037bc admin admin] PATH_INFO: `/v3/projects/ab0e9858e3b7417e99a0e700db8a8d16/users/028c47dddac54a6eb57e66174d1adca9/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:29.311329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e6653-441c-4c15-bd7a-2bb5e96037bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0e9858e3b7417e99a0e700db8a8d16, user_id=028c47dddac54a6eb57e66174d1adca9, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:29.314011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e6653-441c-4c15-bd7a-2bb5e96037bc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:29.384579 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1198/2380] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:29 2022] PUT /identity/v3/projects/ab0e9858e3b7417e99a0e700db8a8d16/users/028c47dddac54a6eb57e66174d1adca9/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:37:29.388594 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0287e54-b14b-4208-ba8e-77f75e0ee819 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:29.399078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0287e54-b14b-4208-ba8e-77f75e0ee819 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:29.418661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:29.454643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5d947-13ea-4ae6-8be3-0681b1e3481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:29.455232 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5d947-13ea-4ae6-8be3-0681b1e3481c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:29.455505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5d947-13ea-4ae6-8be3-0681b1e3481c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:29.455772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5d947-13ea-4ae6-8be3-0681b1e3481c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:29.456426 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5d947-13ea-4ae6-8be3-0681b1e3481c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:29.457994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5d947-13ea-4ae6-8be3-0681b1e3481c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:29.468914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1183/2381] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:29.472958 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e6653-441c-4c15-bd7a-2bb5e96037bc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:29.478348 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:29.483549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e6653-441c-4c15-bd7a-2bb5e96037bc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:29.489304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:29.489304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:29.489304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:29.489304 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] PATH_INFO: `/v3/projects/ab0e9858e3b7417e99a0e700db8a8d16/users/028c47dddac54a6eb57e66174d1adca9/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:29.491101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0e9858e3b7417e99a0e700db8a8d16, user_id=028c47dddac54a6eb57e66174d1adca9, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:29.493790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:29.504124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1199/2382] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:29 2022] PUT /identity/v3/projects/ab0e9858e3b7417e99a0e700db8a8d16/users/028c47dddac54a6eb57e66174d1adca9/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:29.514802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c43aa2-af67-4004-9da0-53461cad05c3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:29.515103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c43aa2-af67-4004-9da0-53461cad05c3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:29.515373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c43aa2-af67-4004-9da0-53461cad05c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:29.542917 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-13c43aa2-af67-4004-9da0-53461cad05c3 None None] MFA Rules not processed for user `028c47dddac54a6eb57e66174d1adca9`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:29.544103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-13c43aa2-af67-4004-9da0-53461cad05c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:29.609820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1184/2383] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:29.626536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:29.628213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:29.692181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71f1b9-2933-4d74-8b97-ea71a9a11e80 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:29.696470 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a06e9-f3bd-4453-89ed-1fe5d901e85a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:29.697076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a06e9-f3bd-4453-89ed-1fe5d901e85a service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:29.697353 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a06e9-f3bd-4453-89ed-1fe5d901e85a service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:29.697622 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a06e9-f3bd-4453-89ed-1fe5d901e85a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:29.708530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a06e9-f3bd-4453-89ed-1fe5d901e85a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:29.710197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a06e9-f3bd-4453-89ed-1fe5d901e85a service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:29.724831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1200/2384] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:29.907509 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c43aa2-af67-4004-9da0-53461cad05c3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:29.917941 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c43aa2-af67-4004-9da0-53461cad05c3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:29.921896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28db542-1998-4baa-90c4-41d6e6de1d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:29.922563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db542-1998-4baa-90c4-41d6e6de1d3e service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:29.922912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db542-1998-4baa-90c4-41d6e6de1d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:29.923256 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db542-1998-4baa-90c4-41d6e6de1d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:29.983484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28db542-1998-4baa-90c4-41d6e6de1d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:29.984969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28db542-1998-4baa-90c4-41d6e6de1d3e service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:29.994721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1185/2385] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:33.290378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a06e9-f3bd-4453-89ed-1fe5d901e85a service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:33.351427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a06e9-f3bd-4453-89ed-1fe5d901e85a service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:33.355932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:33.356601 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:33.356874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:33.357139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] PATH_INFO: `/v3/users/7e1454b53c4844e7b6a9749cfc9cfd42` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:33.358219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e1454b53c4844e7b6a9749cfc9cfd42)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:33.360112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:33.500193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e1454b53c4844e7b6a9749cfc9cfd42'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:33.500891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e1454b53c4844e7b6a9749cfc9cfd42'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:33.513357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e1454b53c4844e7b6a9749cfc9cfd42'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:33.520345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e1454b53c4844e7b6a9749cfc9cfd42'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:33.520934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e1454b53c4844e7b6a9749cfc9cfd42'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:33.573814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1201/2386] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:33 2022] DELETE /identity/v3/users/7e1454b53c4844e7b6a9749cfc9cfd42 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:33.578497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b50b0-2a64-4fcb-815c-8a80c3607d08 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:33.580819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b50b0-2a64-4fcb-815c-8a80c3607d08 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:33.580819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b50b0-2a64-4fcb-815c-8a80c3607d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:33.598192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a35b50b0-2a64-4fcb-815c-8a80c3607d08 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:33.599367 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a35b50b0-2a64-4fcb-815c-8a80c3607d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:33.639524 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:33.657887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1186/2387] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:33.667931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:33.669349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:33.683089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:33.726782 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859cf2b-c5ff-4964-83d7-550f7c6cfa97 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:33.730439 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9d5a-01bc-460d-80fd-bf343332e661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:33.731042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9d5a-01bc-460d-80fd-bf343332e661 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:33.731315 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9d5a-01bc-460d-80fd-bf343332e661 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:33.731584 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa9d5a-01bc-460d-80fd-bf343332e661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:33.740396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa9d5a-01bc-460d-80fd-bf343332e661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:33.742410 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa9d5a-01bc-460d-80fd-bf343332e661 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:33.751529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1202/2388] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:34.163077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b50b0-2a64-4fcb-815c-8a80c3607d08 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:34.165302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a35b50b0-2a64-4fcb-815c-8a80c3607d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:34.177358 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b50b0-2a64-4fcb-815c-8a80c3607d08 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:34.181934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:34.182561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:34.182948 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:34.183234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] PATH_INFO: `/v3/projects/3c95f93cb2414684936cb8df1da00738` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:34.184381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c95f93cb2414684936cb8df1da00738)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:34.187741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:34.321706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c95f93cb2414684936cb8df1da00738', 'request_id': 'req-2497734c-3883-4e12-acdd-cbc522e6aac8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.327792 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c95f93cb2414684936cb8df1da00738', 'request_id': 'req-2497734c-3883-4e12-acdd-cbc522e6aac8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.334115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c95f93cb2414684936cb8df1da00738', 'request_id': 'req-2497734c-3883-4e12-acdd-cbc522e6aac8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.423715 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] The token cache is being invalidate because project 3c95f93cb2414684936cb8df1da00738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:34.423996 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.424630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.426120 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1187/2389] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:34 2022] DELETE /identity/v3/projects/3c95f93cb2414684936cb8df1da00738 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:34.430269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9d5a-01bc-460d-80fd-bf343332e661 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:34.439827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa9d5a-01bc-460d-80fd-bf343332e661 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:34.495625 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:34.496240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:34.496547 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:34.496819 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] PATH_INFO: `/v3/users/028c47dddac54a6eb57e66174d1adca9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:34.497947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=028c47dddac54a6eb57e66174d1adca9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:34.499852 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:34.671982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '028c47dddac54a6eb57e66174d1adca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.672649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '028c47dddac54a6eb57e66174d1adca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.684461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '028c47dddac54a6eb57e66174d1adca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.691585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '028c47dddac54a6eb57e66174d1adca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.692210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-793b603e-e7d9-4178-b9ae-d11d4b37ef07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '028c47dddac54a6eb57e66174d1adca9'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:34.772867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1203/2390] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:34 2022] DELETE /identity/v3/users/028c47dddac54a6eb57e66174d1adca9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:35.195342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:35.195342 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:35.206751 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497734c-3883-4e12-acdd-cbc522e6aac8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:35.265338 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:35.265950 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.266224 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.266488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] PATH_INFO: `/v3/projects/ab0e9858e3b7417e99a0e700db8a8d16` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.267577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0e9858e3b7417e99a0e700db8a8d16)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:35.269481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:35.367789 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0e9858e3b7417e99a0e700db8a8d16', 'request_id': 'req-b379759b-cd2a-4205-adba-06240a14d6d9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:35.373332 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0e9858e3b7417e99a0e700db8a8d16', 'request_id': 'req-b379759b-cd2a-4205-adba-06240a14d6d9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:35.379740 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0e9858e3b7417e99a0e700db8a8d16', 'request_id': 'req-b379759b-cd2a-4205-adba-06240a14d6d9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:35.454020 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:35.458435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] The token cache is being invalidate because project ab0e9858e3b7417e99a0e700db8a8d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:35.464178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:35.464178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b379759b-cd2a-4205-adba-06240a14d6d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:35.464178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1188/2391] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:35 2022] DELETE /identity/v3/projects/ab0e9858e3b7417e99a0e700db8a8d16 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:35.466797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca6b35-538e-4d33-a253-4325e19eb1c3 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.467264 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca6b35-538e-4d33-a253-4325e19eb1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.467395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca6b35-538e-4d33-a253-4325e19eb1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.487181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-52ca6b35-538e-4d33-a253-4325e19eb1c3 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:35.488403 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-52ca6b35-538e-4d33-a253-4325e19eb1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:35.517814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:35.547610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1204/2392] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:35.555357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5a4dd-2e17-48f1-9f83-1fcd4c920e02 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.555641 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5a4dd-2e17-48f1-9f83-1fcd4c920e02 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.555914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5a4dd-2e17-48f1-9f83-1fcd4c920e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.575654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dbe5a4dd-2e17-48f1-9f83-1fcd4c920e02 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:35.576881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dbe5a4dd-2e17-48f1-9f83-1fcd4c920e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:35.585961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1189/2393] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:35.591388 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca6b35-538e-4d33-a253-4325e19eb1c3 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:35.599277 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca6b35-538e-4d33-a253-4325e19eb1c3 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:35.604451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:35.605035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.605324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.605611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.606291 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:37:35.606586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:35.608158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:35.620474 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1205/2394] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:37:35.624697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5a4dd-2e17-48f1-9f83-1fcd4c920e02 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:35.634161 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5a4dd-2e17-48f1-9f83-1fcd4c920e02 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:35.638987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff6d37-416f-494c-a6ad-e47a8376351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:35.639578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff6d37-416f-494c-a6ad-e47a8376351d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.639856 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff6d37-416f-494c-a6ad-e47a8376351d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.640126 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff6d37-416f-494c-a6ad-e47a8376351d admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.640791 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff6d37-416f-494c-a6ad-e47a8376351d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:35.642820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff6d37-416f-494c-a6ad-e47a8376351d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:35.718341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1190/2395] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:35 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:35.722018 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:35.735163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440292d-5312-49ab-a733-55ef4a79d77e admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:35.790987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784f140-903d-4454-9db9-0f8a0c048323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:35.791593 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784f140-903d-4454-9db9-0f8a0c048323 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.791873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784f140-903d-4454-9db9-0f8a0c048323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.792147 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784f140-903d-4454-9db9-0f8a0c048323 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.792816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784f140-903d-4454-9db9-0f8a0c048323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:35.794724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784f140-903d-4454-9db9-0f8a0c048323 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:35.864124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1206/2396] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:35 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:35.868470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff6d37-416f-494c-a6ad-e47a8376351d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:35.878811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff6d37-416f-494c-a6ad-e47a8376351d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:35.883384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51326f34-8256-4fd3-a8d2-d40e4ae54944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:35.883977 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326f34-8256-4fd3-a8d2-d40e4ae54944 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.884245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326f34-8256-4fd3-a8d2-d40e4ae54944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.884557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326f34-8256-4fd3-a8d2-d40e4ae54944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.885198 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51326f34-8256-4fd3-a8d2-d40e4ae54944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:35.886765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51326f34-8256-4fd3-a8d2-d40e4ae54944 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:35.896492 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1191/2397] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:35.900973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784f140-903d-4454-9db9-0f8a0c048323 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:35.910981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784f140-903d-4454-9db9-0f8a0c048323 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:35.916181 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe48ff-9e87-48ea-b021-039f918ad9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:35.916867 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe48ff-9e87-48ea-b021-039f918ad9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.917146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe48ff-9e87-48ea-b021-039f918ad9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.917418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe48ff-9e87-48ea-b021-039f918ad9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.918047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe48ff-9e87-48ea-b021-039f918ad9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:35.919623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe48ff-9e87-48ea-b021-039f918ad9a8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:35.930358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1207/2398] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:35.934412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51326f34-8256-4fd3-a8d2-d40e4ae54944 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:35.944210 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51326f34-8256-4fd3-a8d2-d40e4ae54944 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:35.949649 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee0d83-07af-464b-a5d8-fb03bd5468b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:35.950278 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee0d83-07af-464b-a5d8-fb03bd5468b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:35.954553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee0d83-07af-464b-a5d8-fb03bd5468b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:35.954553 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee0d83-07af-464b-a5d8-fb03bd5468b6 admin admin] PATH_INFO: `/v3/projects/747bb1f182984b91a59751510e926fc4/users/6abe203909da4f5d939232f7ff028806/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:35.966113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee0d83-07af-464b-a5d8-fb03bd5468b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747bb1f182984b91a59751510e926fc4, user_id=6abe203909da4f5d939232f7ff028806, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:35.968824 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee0d83-07af-464b-a5d8-fb03bd5468b6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:36.058522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1192/2399] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:35 2022] PUT /identity/v3/projects/747bb1f182984b91a59751510e926fc4/users/6abe203909da4f5d939232f7ff028806/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:36.069026 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb91d93-a623-4135-a303-7e61b808f373 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.069329 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb91d93-a623-4135-a303-7e61b808f373 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.069596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb91d93-a623-4135-a303-7e61b808f373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.098703 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-eeb91d93-a623-4135-a303-7e61b808f373 None None] MFA Rules not processed for user `6abe203909da4f5d939232f7ff028806`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:36.099896 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eeb91d93-a623-4135-a303-7e61b808f373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:36.163446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1208/2400] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:36.168114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee0d83-07af-464b-a5d8-fb03bd5468b6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:36.228340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee0d83-07af-464b-a5d8-fb03bd5468b6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:36.232814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310feeaa-9184-4c3e-ac77-74b726e637c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:36.233429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310feeaa-9184-4c3e-ac77-74b726e637c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.233705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310feeaa-9184-4c3e-ac77-74b726e637c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.233971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310feeaa-9184-4c3e-ac77-74b726e637c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.234597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310feeaa-9184-4c3e-ac77-74b726e637c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:36.236499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310feeaa-9184-4c3e-ac77-74b726e637c5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:36.312226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1193/2401] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:36 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:36.316415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb91d93-a623-4135-a303-7e61b808f373 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:36.327800 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb91d93-a623-4135-a303-7e61b808f373 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:36.353702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:36.385000 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136013c-8b46-4689-9a7d-c91924716b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:36.385612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136013c-8b46-4689-9a7d-c91924716b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.386216 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136013c-8b46-4689-9a7d-c91924716b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.386567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136013c-8b46-4689-9a7d-c91924716b12 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.387205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136013c-8b46-4689-9a7d-c91924716b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:36.389175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136013c-8b46-4689-9a7d-c91924716b12 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:36.464537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1209/2402] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:36 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:36.468493 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310feeaa-9184-4c3e-ac77-74b726e637c5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:36.478759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310feeaa-9184-4c3e-ac77-74b726e637c5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:36.483545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf3e44-25ea-4291-92a4-6b7c6b8f1801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:36.484137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf3e44-25ea-4291-92a4-6b7c6b8f1801 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.484454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf3e44-25ea-4291-92a4-6b7c6b8f1801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.484730 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf3e44-25ea-4291-92a4-6b7c6b8f1801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.485360 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf3e44-25ea-4291-92a4-6b7c6b8f1801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:36.486963 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf3e44-25ea-4291-92a4-6b7c6b8f1801 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:36.498165 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1194/2403] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:36.502040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136013c-8b46-4689-9a7d-c91924716b12 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:36.512841 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136013c-8b46-4689-9a7d-c91924716b12 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:36.517506 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9544aa6-7a65-4898-b36a-472dbcbdd5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:36.518153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544aa6-7a65-4898-b36a-472dbcbdd5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.518427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544aa6-7a65-4898-b36a-472dbcbdd5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.519016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544aa6-7a65-4898-b36a-472dbcbdd5a9 admin admin] PATH_INFO: `/v3/projects/d39ec4363cf7468582d45d3906a6cf90/users/4604bd76461b4a148fa2a6f34feaaca0/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.533532 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9544aa6-7a65-4898-b36a-472dbcbdd5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39ec4363cf7468582d45d3906a6cf90, user_id=4604bd76461b4a148fa2a6f34feaaca0, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:36.536156 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9544aa6-7a65-4898-b36a-472dbcbdd5a9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:36.590114 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1210/2404] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:36 2022] PUT /identity/v3/projects/d39ec4363cf7468582d45d3906a6cf90/users/4604bd76461b4a148fa2a6f34feaaca0/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:36.594023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf3e44-25ea-4291-92a4-6b7c6b8f1801 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:36.604073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf3e44-25ea-4291-92a4-6b7c6b8f1801 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:36.659121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9f698-883c-41e9-bfdb-be412f29ed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:36.659713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f698-883c-41e9-bfdb-be412f29ed6a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.659987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f698-883c-41e9-bfdb-be412f29ed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.660253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9f698-883c-41e9-bfdb-be412f29ed6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.660901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9f698-883c-41e9-bfdb-be412f29ed6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:36.662462 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9f698-883c-41e9-bfdb-be412f29ed6a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:36.672810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1195/2405] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:36.676817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9544aa6-7a65-4898-b36a-472dbcbdd5a9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:36.687375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9544aa6-7a65-4898-b36a-472dbcbdd5a9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:36.691840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:36.692612 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.692886 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.693155 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] PATH_INFO: `/v3/projects/d39ec4363cf7468582d45d3906a6cf90/users/4604bd76461b4a148fa2a6f34feaaca0/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.695019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39ec4363cf7468582d45d3906a6cf90, user_id=4604bd76461b4a148fa2a6f34feaaca0, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:36.697838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:36.707951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1211/2406] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:36 2022] PUT /identity/v3/projects/d39ec4363cf7468582d45d3906a6cf90/users/4604bd76461b4a148fa2a6f34feaaca0/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:36.718842 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56eb75-db4e-4d94-b9ad-13485c0d816b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.719124 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56eb75-db4e-4d94-b9ad-13485c0d816b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.719397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56eb75-db4e-4d94-b9ad-13485c0d816b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.727828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:36.748156 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dc56eb75-db4e-4d94-b9ad-13485c0d816b None None] MFA Rules not processed for user `4604bd76461b4a148fa2a6f34feaaca0`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:36.749346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dc56eb75-db4e-4d94-b9ad-13485c0d816b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:36.818795 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1196/2407] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:36.830327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:36.831956 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:36.887866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3a963-7324-41a1-a446-3a617a7cd591 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:36.891428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63031d92-7997-4ec2-ac24-50bd65e6b314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:36.892021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031d92-7997-4ec2-ac24-50bd65e6b314 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:36.892290 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031d92-7997-4ec2-ac24-50bd65e6b314 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:36.892580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031d92-7997-4ec2-ac24-50bd65e6b314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:36.901256 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63031d92-7997-4ec2-ac24-50bd65e6b314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:36.902683 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63031d92-7997-4ec2-ac24-50bd65e6b314 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:36.911793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1212/2408] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:36.979294 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56eb75-db4e-4d94-b9ad-13485c0d816b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:37.041853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56eb75-db4e-4d94-b9ad-13485c0d816b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:37.046293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:37.046922 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:37.047202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:37.047475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] PATH_INFO: `/v3/users/6abe203909da4f5d939232f7ff028806` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:37.048625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abe203909da4f5d939232f7ff028806)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:37.050534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:37.209109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abe203909da4f5d939232f7ff028806'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:37.216895 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abe203909da4f5d939232f7ff028806'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:37.229777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abe203909da4f5d939232f7ff028806'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:37.315784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abe203909da4f5d939232f7ff028806'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:37.316364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abe203909da4f5d939232f7ff028806'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:37.322477 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1197/2409] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:36 2022] DELETE /identity/v3/users/6abe203909da4f5d939232f7ff028806 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:37.327269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feb71a-d74c-4e1b-9df1-87338a49830f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:37.327549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feb71a-d74c-4e1b-9df1-87338a49830f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:37.327816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feb71a-d74c-4e1b-9df1-87338a49830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:37.346671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-18feb71a-d74c-4e1b-9df1-87338a49830f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:37.347847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-18feb71a-d74c-4e1b-9df1-87338a49830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:37.404922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1213/2410] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:37.415150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:37.416561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:37.427295 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:37.473841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68361d59-feb5-48d1-9eea-49a95e449282 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:37.477442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e781cdd-8d2b-42e8-85c2-f2067b046911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:37.478032 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e781cdd-8d2b-42e8-85c2-f2067b046911 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:37.478309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e781cdd-8d2b-42e8-85c2-f2067b046911 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:37.478580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e781cdd-8d2b-42e8-85c2-f2067b046911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:37.487987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e781cdd-8d2b-42e8-85c2-f2067b046911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:37.489431 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e781cdd-8d2b-42e8-85c2-f2067b046911 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:37.498783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1198/2411] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:37:37.865525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18feb71a-d74c-4e1b-9df1-87338a49830f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:37.871825 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-18feb71a-d74c-4e1b-9df1-87338a49830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:37.879754 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18feb71a-d74c-4e1b-9df1-87338a49830f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:37.885040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:37.885702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:37.886091 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:37.886368 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] PATH_INFO: `/v3/projects/747bb1f182984b91a59751510e926fc4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:37.887494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747bb1f182984b91a59751510e926fc4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:37.889434 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:37.911555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:38.019902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747bb1f182984b91a59751510e926fc4', 'request_id': 'req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.093075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747bb1f182984b91a59751510e926fc4', 'request_id': 'req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.099945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747bb1f182984b91a59751510e926fc4', 'request_id': 'req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.114186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] The token cache is being invalidate because project 747bb1f182984b91a59751510e926fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:38.114186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.114186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.114186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1214/2412] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:37 2022] DELETE /identity/v3/projects/747bb1f182984b91a59751510e926fc4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:38.117405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e781cdd-8d2b-42e8-85c2-f2067b046911 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:38.126899 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e781cdd-8d2b-42e8-85c2-f2067b046911 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:38.184088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:38.184896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:38.185181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:38.185458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] PATH_INFO: `/v3/users/4604bd76461b4a148fa2a6f34feaaca0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:38.186576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4604bd76461b4a148fa2a6f34feaaca0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:38.188575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:38.403246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4604bd76461b4a148fa2a6f34feaaca0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.410284 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4604bd76461b4a148fa2a6f34feaaca0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.422690 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4604bd76461b4a148fa2a6f34feaaca0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.493520 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4604bd76461b4a148fa2a6f34feaaca0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.494122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e1d0026a-f304-490e-9eb5-bc05d5038232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4604bd76461b4a148fa2a6f34feaaca0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:38.499934 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1199/2413] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:38 2022] DELETE /identity/v3/users/4604bd76461b4a148fa2a6f34feaaca0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:38.865659 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:38.867157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:38.883286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c1bbf-2f1e-4df5-bb73-a6ea0d7267f7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:38.940137 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:38.940716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:38.940992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:38.941261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] PATH_INFO: `/v3/projects/d39ec4363cf7468582d45d3906a6cf90` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:38.942347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39ec4363cf7468582d45d3906a6cf90)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:38.944234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:39.058083 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39ec4363cf7468582d45d3906a6cf90', 'request_id': 'req-8e7c2ce0-769e-437b-be0d-511295d604eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:39.127046 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39ec4363cf7468582d45d3906a6cf90', 'request_id': 'req-8e7c2ce0-769e-437b-be0d-511295d604eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:39.134363 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39ec4363cf7468582d45d3906a6cf90', 'request_id': 'req-8e7c2ce0-769e-437b-be0d-511295d604eb'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:39.144633 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] The token cache is being invalidate because project d39ec4363cf7468582d45d3906a6cf90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:39.144915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:39.145533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8e7c2ce0-769e-437b-be0d-511295d604eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:39.146997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1215/2414] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:38 2022] DELETE /identity/v3/projects/d39ec4363cf7468582d45d3906a6cf90 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:39.152543 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a4f7-6df9-48d0-a5f8-bfe58377024d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.152834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a4f7-6df9-48d0-a5f8-bfe58377024d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.153142 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7a4f7-6df9-48d0-a5f8-bfe58377024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.173786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4ac7a4f7-6df9-48d0-a5f8-bfe58377024d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:39.174984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4ac7a4f7-6df9-48d0-a5f8-bfe58377024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:39.201774 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:39.233533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1200/2415] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:39.241426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c2bfb-5608-4c7e-b324-f578940626be None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.241715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c2bfb-5608-4c7e-b324-f578940626be None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.241997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c2bfb-5608-4c7e-b324-f578940626be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.261759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-102c2bfb-5608-4c7e-b324-f578940626be None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:39.262950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-102c2bfb-5608-4c7e-b324-f578940626be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:39.270695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1216/2416] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:39.275397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7a4f7-6df9-48d0-a5f8-bfe58377024d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:39.285169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7a4f7-6df9-48d0-a5f8-bfe58377024d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:39.289561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:39.294239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.294239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.294239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.294239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:37:39.294239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:39.294239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:39.305058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1201/2417] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:37:39.309001 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c2bfb-5608-4c7e-b324-f578940626be None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:39.317950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c2bfb-5608-4c7e-b324-f578940626be None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:39.322997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62114-3150-4739-bc82-bab97d95e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:39.323566 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62114-3150-4739-bc82-bab97d95e1cc admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.324085 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62114-3150-4739-bc82-bab97d95e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.324395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62114-3150-4739-bc82-bab97d95e1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.325020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe62114-3150-4739-bc82-bab97d95e1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:39.326875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe62114-3150-4739-bc82-bab97d95e1cc admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:39.391086 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1217/2418] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:39 2022] POST /identity/v3/projects => generated 387 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:39.395376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:39.405331 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e998029-29f7-422d-b05b-4865eb18cf7e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:39.460327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5664445-abf5-4d4b-bda3-053d79b49b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:39.460850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5664445-abf5-4d4b-bda3-053d79b49b62 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.461124 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5664445-abf5-4d4b-bda3-053d79b49b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.461394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5664445-abf5-4d4b-bda3-053d79b49b62 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.462003 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5664445-abf5-4d4b-bda3-053d79b49b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:39.463871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5664445-abf5-4d4b-bda3-053d79b49b62 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:39.561394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1202/2419] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:39 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:39.565496 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62114-3150-4739-bc82-bab97d95e1cc admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:39.576689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62114-3150-4739-bc82-bab97d95e1cc admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:39.581326 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674dfa6-bcc7-4e1f-9415-8d25071ba068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:39.581962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674dfa6-bcc7-4e1f-9415-8d25071ba068 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.582253 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674dfa6-bcc7-4e1f-9415-8d25071ba068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.582525 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674dfa6-bcc7-4e1f-9415-8d25071ba068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.583160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674dfa6-bcc7-4e1f-9415-8d25071ba068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:39.584774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674dfa6-bcc7-4e1f-9415-8d25071ba068 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:39.587414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:39.595849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1218/2420] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:39.599880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5664445-abf5-4d4b-bda3-053d79b49b62 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:39.609706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5664445-abf5-4d4b-bda3-053d79b49b62 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:39.614253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e463-8aaf-4b95-9a63-9494fb9d2e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:39.614834 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e463-8aaf-4b95-9a63-9494fb9d2e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.615107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e463-8aaf-4b95-9a63-9494fb9d2e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.615377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e463-8aaf-4b95-9a63-9494fb9d2e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.616002 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09e463-8aaf-4b95-9a63-9494fb9d2e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:39.618132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09e463-8aaf-4b95-9a63-9494fb9d2e8c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:39.628351 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1203/2421] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:37:39.633191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674dfa6-bcc7-4e1f-9415-8d25071ba068 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:39.642785 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674dfa6-bcc7-4e1f-9415-8d25071ba068 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:39.647362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6073816-87fc-4098-b938-829864b91f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:39.648019 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6073816-87fc-4098-b938-829864b91f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.648312 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6073816-87fc-4098-b938-829864b91f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.648586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6073816-87fc-4098-b938-829864b91f8d admin admin] PATH_INFO: `/v3/projects/3f4a5e50975b44f5bc3424ff2afdf5e3/users/e141c62845de4edfbf69823a55e84b70/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.665505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6073816-87fc-4098-b938-829864b91f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f4a5e50975b44f5bc3424ff2afdf5e3, user_id=e141c62845de4edfbf69823a55e84b70, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:39.668283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6073816-87fc-4098-b938-829864b91f8d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:39.761998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1219/2422] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:39 2022] PUT /identity/v3/projects/3f4a5e50975b44f5bc3424ff2afdf5e3/users/e141c62845de4edfbf69823a55e84b70/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:39.772375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09d818-536f-43bb-8c60-fe07840eef23 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.772663 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09d818-536f-43bb-8c60-fe07840eef23 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.772937 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09d818-536f-43bb-8c60-fe07840eef23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.800828 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ee09d818-536f-43bb-8c60-fe07840eef23 None None] MFA Rules not processed for user `e141c62845de4edfbf69823a55e84b70`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:39.802053 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ee09d818-536f-43bb-8c60-fe07840eef23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:39.866062 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1204/2423] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:39.871127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6073816-87fc-4098-b938-829864b91f8d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:39.930699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6073816-87fc-4098-b938-829864b91f8d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:39.935235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abee795-daec-485d-8cf2-e1c8f4ef60c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:39.935817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee795-daec-485d-8cf2-e1c8f4ef60c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:39.936090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee795-daec-485d-8cf2-e1c8f4ef60c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:39.936386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abee795-daec-485d-8cf2-e1c8f4ef60c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:39.936997 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abee795-daec-485d-8cf2-e1c8f4ef60c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:39.938859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abee795-daec-485d-8cf2-e1c8f4ef60c2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.008466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1220/2424] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:39 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:40.012489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09d818-536f-43bb-8c60-fe07840eef23 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.023674 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09d818-536f-43bb-8c60-fe07840eef23 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:40.048488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:40.080471 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49112e-c7d6-4c94-b9d2-5fcb8bca5580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:40.081082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49112e-c7d6-4c94-b9d2-5fcb8bca5580 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.081378 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49112e-c7d6-4c94-b9d2-5fcb8bca5580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.081794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49112e-c7d6-4c94-b9d2-5fcb8bca5580 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.082429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49112e-c7d6-4c94-b9d2-5fcb8bca5580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:40.084371 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49112e-c7d6-4c94-b9d2-5fcb8bca5580 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.155766 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1205/2425] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:40 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:40.159977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abee795-daec-485d-8cf2-e1c8f4ef60c2 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.170672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abee795-daec-485d-8cf2-e1c8f4ef60c2 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:40.175123 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b84ff8-6b84-437d-b014-2e8a9ed9ab6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:40.175699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84ff8-6b84-437d-b014-2e8a9ed9ab6c admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.175974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84ff8-6b84-437d-b014-2e8a9ed9ab6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.176243 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84ff8-6b84-437d-b014-2e8a9ed9ab6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.176895 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b84ff8-6b84-437d-b014-2e8a9ed9ab6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:40.178522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b84ff8-6b84-437d-b014-2e8a9ed9ab6c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.188818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1221/2426] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:40.192578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49112e-c7d6-4c94-b9d2-5fcb8bca5580 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.202669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49112e-c7d6-4c94-b9d2-5fcb8bca5580 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:40.207204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48370196-2730-44fe-bad9-a2e1e2f903cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:40.207781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48370196-2730-44fe-bad9-a2e1e2f903cb admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.208057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48370196-2730-44fe-bad9-a2e1e2f903cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.208359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48370196-2730-44fe-bad9-a2e1e2f903cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.208991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48370196-2730-44fe-bad9-a2e1e2f903cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:40.211077 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48370196-2730-44fe-bad9-a2e1e2f903cb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.220782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1206/2427] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:40.224636 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b84ff8-6b84-437d-b014-2e8a9ed9ab6c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.234821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b84ff8-6b84-437d-b014-2e8a9ed9ab6c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:40.239325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a0d3b-5414-489e-b5dc-9a84c932202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:40.239945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a0d3b-5414-489e-b5dc-9a84c932202d admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.240219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a0d3b-5414-489e-b5dc-9a84c932202d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.240522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a0d3b-5414-489e-b5dc-9a84c932202d admin admin] PATH_INFO: `/v3/projects/91f128d933ee495291ef7504195b199a/users/298958b5baad4c988b9a6ff146f8d34e/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.254358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a0d3b-5414-489e-b5dc-9a84c932202d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f128d933ee495291ef7504195b199a, user_id=298958b5baad4c988b9a6ff146f8d34e, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:40.257036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a0d3b-5414-489e-b5dc-9a84c932202d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.327006 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1222/2428] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:40 2022] PUT /identity/v3/projects/91f128d933ee495291ef7504195b199a/users/298958b5baad4c988b9a6ff146f8d34e/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:40.391769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca55a-794a-4ccc-8c02-c24c0b75a03a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.392054 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca55a-794a-4ccc-8c02-c24c0b75a03a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.392382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca55a-794a-4ccc-8c02-c24c0b75a03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.420586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-610ca55a-794a-4ccc-8c02-c24c0b75a03a None None] MFA Rules not processed for user `298958b5baad4c988b9a6ff146f8d34e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:40.421778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-610ca55a-794a-4ccc-8c02-c24c0b75a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:40.484171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1207/2429] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:40.488649 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a0d3b-5414-489e-b5dc-9a84c932202d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.548784 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a0d3b-5414-489e-b5dc-9a84c932202d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:40.553482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590448dd-c898-480d-93e4-f5c835b18c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:40.554146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590448dd-c898-480d-93e4-f5c835b18c20 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.554425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590448dd-c898-480d-93e4-f5c835b18c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.554696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590448dd-c898-480d-93e4-f5c835b18c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.555316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590448dd-c898-480d-93e4-f5c835b18c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:40.557226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590448dd-c898-480d-93e4-f5c835b18c20 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.688236 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1223/2430] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:37:40 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:37:40.692242 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca55a-794a-4ccc-8c02-c24c0b75a03a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.704107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca55a-794a-4ccc-8c02-c24c0b75a03a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:40.759123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9eac7-75d6-4c67-bed0-48cd114c973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:40.759726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9eac7-75d6-4c67-bed0-48cd114c973c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.760013 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9eac7-75d6-4c67-bed0-48cd114c973c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.760312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9eac7-75d6-4c67-bed0-48cd114c973c admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.760984 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9eac7-75d6-4c67-bed0-48cd114c973c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:40.762986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9eac7-75d6-4c67-bed0-48cd114c973c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.859057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1208/2431] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:37:40 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:37:40.863125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590448dd-c898-480d-93e4-f5c835b18c20 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.873628 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590448dd-c898-480d-93e4-f5c835b18c20 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:40.878594 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37584f-4916-4b9e-ad3b-837242490b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:40.879185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37584f-4916-4b9e-ad3b-837242490b54 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.879466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37584f-4916-4b9e-ad3b-837242490b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.879740 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37584f-4916-4b9e-ad3b-837242490b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.880426 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37584f-4916-4b9e-ad3b-837242490b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:40.883489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37584f-4916-4b9e-ad3b-837242490b54 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.884111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:40.892910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1224/2432] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:37:40.897301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9eac7-75d6-4c67-bed0-48cd114c973c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.907575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9eac7-75d6-4c67-bed0-48cd114c973c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:40.912121 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5656392-f9e1-4a5c-975d-cb1c161694f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:40.912811 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656392-f9e1-4a5c-975d-cb1c161694f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:40.913089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656392-f9e1-4a5c-975d-cb1c161694f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:40.913361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656392-f9e1-4a5c-975d-cb1c161694f9 admin admin] PATH_INFO: `/v3/projects/46e1dcfbe01241dfb9eebffba376b09f/users/ca56dac19c39422490a1c9edb9ad7d68/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:40.928561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5656392-f9e1-4a5c-975d-cb1c161694f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e1dcfbe01241dfb9eebffba376b09f, user_id=ca56dac19c39422490a1c9edb9ad7d68, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:40.931189 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5656392-f9e1-4a5c-975d-cb1c161694f9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:40.974219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:40.979219 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1209/2433] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:40 2022] PUT /identity/v3/projects/46e1dcfbe01241dfb9eebffba376b09f/users/ca56dac19c39422490a1c9edb9ad7d68/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:37:40.983150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37584f-4916-4b9e-ad3b-837242490b54 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:40.993009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37584f-4916-4b9e-ad3b-837242490b54 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:41.048376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0d7e-9966-4f24-8af8-2719fc2102e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:41.048979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0d7e-9966-4f24-8af8-2719fc2102e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:41.049261 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0d7e-9966-4f24-8af8-2719fc2102e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:41.049533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0d7e-9966-4f24-8af8-2719fc2102e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:41.050167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0d7e-9966-4f24-8af8-2719fc2102e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:41.051778 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0d7e-9966-4f24-8af8-2719fc2102e2 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:41.061673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1225/2434] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:37:41.065955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5656392-f9e1-4a5c-975d-cb1c161694f9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:41.076475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5656392-f9e1-4a5c-975d-cb1c161694f9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:41.081000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:41.081637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:41.081911 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:41.082180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] PATH_INFO: `/v3/projects/46e1dcfbe01241dfb9eebffba376b09f/users/ca56dac19c39422490a1c9edb9ad7d68/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:41.084047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e1dcfbe01241dfb9eebffba376b09f, user_id=ca56dac19c39422490a1c9edb9ad7d68, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:41.087271 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:41.097822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1210/2435] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:37:41 2022] PUT /identity/v3/projects/46e1dcfbe01241dfb9eebffba376b09f/users/ca56dac19c39422490a1c9edb9ad7d68/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:37:41.108694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f6d6-6944-40f6-a69b-a900ea7669d4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:41.113242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f6d6-6944-40f6-a69b-a900ea7669d4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:41.113242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f6d6-6944-40f6-a69b-a900ea7669d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:41.136424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a6e7f6d6-6944-40f6-a69b-a900ea7669d4 None None] MFA Rules not processed for user `ca56dac19c39422490a1c9edb9ad7d68`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:41.137614 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a6e7f6d6-6944-40f6-a69b-a900ea7669d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:41.205955 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1226/2436] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:41.217481 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:41.219153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:41.277266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155009ed-4029-4b0e-9162-9e0ef167bec8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:41.280836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bba51-0234-4ae2-83d2-492c57a152d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:41.281423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bba51-0234-4ae2-83d2-492c57a152d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:41.281699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bba51-0234-4ae2-83d2-492c57a152d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:41.281968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bba51-0234-4ae2-83d2-492c57a152d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:41.290959 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bba51-0234-4ae2-83d2-492c57a152d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:41.292439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bba51-0234-4ae2-83d2-492c57a152d6 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:41.301647 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1211/2437] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:42.339258 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7f6d6-6944-40f6-a69b-a900ea7669d4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:42.340922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a6e7f6d6-6944-40f6-a69b-a900ea7669d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:42.352621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7f6d6-6944-40f6-a69b-a900ea7669d4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:42.356232 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9651c-5bc6-40b2-8679-f4e9abf39226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:42.356838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9651c-5bc6-40b2-8679-f4e9abf39226 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:42.357112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9651c-5bc6-40b2-8679-f4e9abf39226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:42.357376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9651c-5bc6-40b2-8679-f4e9abf39226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:42.412242 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9651c-5bc6-40b2-8679-f4e9abf39226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:42.413858 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9651c-5bc6-40b2-8679-f4e9abf39226 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:42.422473 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1227/2438] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:47.251040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bba51-0234-4ae2-83d2-492c57a152d6 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:47.260339 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bba51-0234-4ae2-83d2-492c57a152d6 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:47.263919 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db71ae-6d7c-492b-8daf-4cd04f4cff5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:47.264540 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db71ae-6d7c-492b-8daf-4cd04f4cff5e service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:47.264819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db71ae-6d7c-492b-8daf-4cd04f4cff5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:47.265094 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db71ae-6d7c-492b-8daf-4cd04f4cff5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:47.320914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db71ae-6d7c-492b-8daf-4cd04f4cff5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:47.322837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db71ae-6d7c-492b-8daf-4cd04f4cff5e service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:47.332096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1212/2439] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:50.352420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59da2-64d5-4206-9fc9-9395b0b31631 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:50.353412 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59da2-64d5-4206-9fc9-9395b0b31631 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:50.353530 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d59da2-64d5-4206-9fc9-9395b0b31631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:50.372976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-32d59da2-64d5-4206-9fc9-9395b0b31631 None None] MFA Rules not processed for user `cbdf62fbe49d44a58c294ac2c8b22ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:50.374149 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-32d59da2-64d5-4206-9fc9-9395b0b31631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:50.385673 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=100 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:50.427186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1228/2440] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:51.137986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db71ae-6d7c-492b-8daf-4cd04f4cff5e service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:51.201401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db71ae-6d7c-492b-8daf-4cd04f4cff5e service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:51.206057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:51.206664 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:51.206942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:51.207216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] PATH_INFO: `/v3/users/e141c62845de4edfbf69823a55e84b70` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:51.208366 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e141c62845de4edfbf69823a55e84b70)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:51.215877 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:51.433578 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:51.449235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e141c62845de4edfbf69823a55e84b70'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:51.456838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e141c62845de4edfbf69823a55e84b70'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:51.471768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e141c62845de4edfbf69823a55e84b70'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:51.556621 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e141c62845de4edfbf69823a55e84b70'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:51.557346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e141c62845de4edfbf69823a55e84b70'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:51.564091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1213/2441] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:51 2022] DELETE /identity/v3/users/e141c62845de4edfbf69823a55e84b70 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:51.568767 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af046563-e26e-42b0-9db5-3d8bec549dfb None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:51.569274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af046563-e26e-42b0-9db5-3d8bec549dfb None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:51.569382 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af046563-e26e-42b0-9db5-3d8bec549dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:51.589921 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-af046563-e26e-42b0-9db5-3d8bec549dfb None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:37:51.591097 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-af046563-e26e-42b0-9db5-3d8bec549dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:51.648037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1229/2442] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:37:51.657871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:51.659265 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:51.718434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f252e-3c76-4362-8e8d-3ba1e911b9cb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:51.722140 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d22398-41eb-47a8-bfdf-a649d38bd888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:51.722738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d22398-41eb-47a8-bfdf-a649d38bd888 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:51.723011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d22398-41eb-47a8-bfdf-a649d38bd888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:51.723276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d22398-41eb-47a8-bfdf-a649d38bd888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:51.733041 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d22398-41eb-47a8-bfdf-a649d38bd888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:51.734587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d22398-41eb-47a8-bfdf-a649d38bd888 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:51.744409 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1214/2443] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:37:52.163562 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af046563-e26e-42b0-9db5-3d8bec549dfb None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:52.165079 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-af046563-e26e-42b0-9db5-3d8bec549dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:52.178721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af046563-e26e-42b0-9db5-3d8bec549dfb None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:52.183346 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:52.184184 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:52.184504 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:52.184795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] PATH_INFO: `/v3/projects/3f4a5e50975b44f5bc3424ff2afdf5e3` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:52.185909 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f4a5e50975b44f5bc3424ff2afdf5e3)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:52.187795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:52.304541 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f4a5e50975b44f5bc3424ff2afdf5e3', 'request_id': 'req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.389585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f4a5e50975b44f5bc3424ff2afdf5e3', 'request_id': 'req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.396469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f4a5e50975b44f5bc3424ff2afdf5e3', 'request_id': 'req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.407414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] The token cache is being invalidate because project 3f4a5e50975b44f5bc3424ff2afdf5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:52.407702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.408365 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.409865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1230/2444] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:52 2022] DELETE /identity/v3/projects/3f4a5e50975b44f5bc3424ff2afdf5e3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:52.414103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d22398-41eb-47a8-bfdf-a649d38bd888 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:52.423987 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d22398-41eb-47a8-bfdf-a649d38bd888 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:52.481359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:52.577628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:52.577628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:52.577628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] PATH_INFO: `/v3/users/298958b5baad4c988b9a6ff146f8d34e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:52.577628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=298958b5baad4c988b9a6ff146f8d34e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:52.577628 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:52.702618 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298958b5baad4c988b9a6ff146f8d34e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.707115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:52.710143 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298958b5baad4c988b9a6ff146f8d34e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.722819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298958b5baad4c988b9a6ff146f8d34e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.784171 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298958b5baad4c988b9a6ff146f8d34e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.784836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298958b5baad4c988b9a6ff146f8d34e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:52.790915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1215/2445] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:52 2022] DELETE /identity/v3/users/298958b5baad4c988b9a6ff146f8d34e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:53.169425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:53.171764 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:53.186271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b5da4-a40e-4b0b-b7e7-3e7b920cb452 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:53.229737 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:37:53.241795 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:53.242417 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:53.242695 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:53.242965 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] PATH_INFO: `/v3/projects/91f128d933ee495291ef7504195b199a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:53.244702 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f128d933ee495291ef7504195b199a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:53.246598 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:53.345209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f128d933ee495291ef7504195b199a', 'request_id': 'req-6a1c5251-c68d-41c3-b20d-93c6f5aef809'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.453065 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f128d933ee495291ef7504195b199a', 'request_id': 'req-6a1c5251-c68d-41c3-b20d-93c6f5aef809'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.459931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f128d933ee495291ef7504195b199a', 'request_id': 'req-6a1c5251-c68d-41c3-b20d-93c6f5aef809'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.471182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] The token cache is being invalidate because project 91f128d933ee495291ef7504195b199a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:53.471469 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.472133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.473642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1231/2446] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:53 2022] DELETE /identity/v3/projects/91f128d933ee495291ef7504195b199a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:53.477830 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:53.479989 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:53.494097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c94f4-6e95-484b-9a85-5f514ffe28e4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:53.553268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:53.553900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:53.554268 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:53.554606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] PATH_INFO: `/v3/users/ca56dac19c39422490a1c9edb9ad7d68` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:53.555755 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca56dac19c39422490a1c9edb9ad7d68)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:53.557700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:53.708913 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca56dac19c39422490a1c9edb9ad7d68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.732965 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca56dac19c39422490a1c9edb9ad7d68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.745619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca56dac19c39422490a1c9edb9ad7d68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.801172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca56dac19c39422490a1c9edb9ad7d68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.801917 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca56dac19c39422490a1c9edb9ad7d68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:53.808983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1216/2447] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:37:53 2022] DELETE /identity/v3/users/ca56dac19c39422490a1c9edb9ad7d68 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:37:54.376745 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:37:54.378358 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:37:54.391759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c5251-c68d-41c3-b20d-93c6f5aef809 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:37:54.460688 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:37:54.465678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:37:54.465678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:37:54.465678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] PATH_INFO: `/v3/projects/46e1dcfbe01241dfb9eebffba376b09f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:37:54.465678 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e1dcfbe01241dfb9eebffba376b09f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:37:54.467077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:37:54.581640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e1dcfbe01241dfb9eebffba376b09f', 'request_id': 'req-ddad24f7-055f-412f-821b-2855e5a86334'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:54.641125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e1dcfbe01241dfb9eebffba376b09f', 'request_id': 'req-ddad24f7-055f-412f-821b-2855e5a86334'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:54.648288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e1dcfbe01241dfb9eebffba376b09f', 'request_id': 'req-ddad24f7-055f-412f-821b-2855e5a86334'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:54.662615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] The token cache is being invalidate because project 46e1dcfbe01241dfb9eebffba376b09f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:37:54.662901 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:54.663631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-ddad24f7-055f-412f-821b-2855e5a86334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:37:54.666637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1232/2448] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:37:54 2022] DELETE /identity/v3/projects/46e1dcfbe01241dfb9eebffba376b09f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:38:03.804676 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=63,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:04.660870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:12.528640 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:12.530172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:12.613771 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b70746-ab70-4f6c-ae4b-ce95256b1152 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:12.684833 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:12.686049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:12.686397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:12.686744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:12.688697 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:12.703359 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:12.705951 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:12.719487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1217/2449] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:38:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:38:13.721849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef475dc-ca38-4248-bd42-0a55fbb58604 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:13.722475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef475dc-ca38-4248-bd42-0a55fbb58604 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:13.722475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef475dc-ca38-4248-bd42-0a55fbb58604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:13.741353 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-eef475dc-ca38-4248-bd42-0a55fbb58604 None None] MFA Rules not processed for user `54fcced6d8d943608b940d1dab001540`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:38:13.742538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eef475dc-ca38-4248-bd42-0a55fbb58604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:13.795824 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1233/2450] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:38:14.098639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:14.100861 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:14.156186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea08e1-7cd5-45d7-9eb1-43f4e201a19a service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:14.159978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:14.160546 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:14.160825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:14.161096 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:14.162897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:14.173067 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:14.174484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:14.182442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1218/2451] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:38:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:38:23.789453 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=25 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:24.178826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:34.917199 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef475dc-ca38-4248-bd42-0a55fbb58604 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:34.918629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-eef475dc-ca38-4248-bd42-0a55fbb58604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:34.987710 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef475dc-ca38-4248-bd42-0a55fbb58604 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:34.992650 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:34.993244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:34.993522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:34.993792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:34.995491 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:35.007151 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:35.008759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:35.019201 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1234/2452] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:38:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:38:45.014451 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=32 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:38:50.154975 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:50.156421 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:50.166877 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18531ecd-735a-40b1-8c12-89caae322ae7 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:50.170467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:50.171059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:50.171341 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:50.171633 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:50.173286 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:50.232544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:50.233962 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:50.242878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1219/2453] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:38:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:38:56.493175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:56.502494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f72e4-4523-4fb5-9d4d-f373679f9440 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:38:56.506660 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:38:56.507245 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:38:56.507527 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:38:56.507797 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:38:56.509465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:38:56.570136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:38:56.571578 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:38:56.581191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1235/2454] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:38:59.986401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:38:59.999579 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc405e9c-aa31-44c9-b535-e2c06cca02c6 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:00.005259 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:00.005972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:00.006253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:00.006523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:00.008447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:00.020373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:39:00.021983 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:39:00.032204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1220/2455] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:38:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:39:03.413820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:39:03.415279 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:03.428435 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63af15a-8bab-4579-804d-eb09e5205575 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:03.432977 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:03.433544 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:03.433823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:03.434090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] PATH_INFO: `/v3/users/33af9a4e3c29426b90dfd5abebde5ed4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:03.435205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33af9a4e3c29426b90dfd5abebde5ed4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:39:03.437126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:39:03.631227 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33af9a4e3c29426b90dfd5abebde5ed4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:03.631907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33af9a4e3c29426b90dfd5abebde5ed4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:03.644620 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33af9a4e3c29426b90dfd5abebde5ed4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:03.651661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33af9a4e3c29426b90dfd5abebde5ed4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:03.652344 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33af9a4e3c29426b90dfd5abebde5ed4'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:03.736529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1236/2456] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:39:03 2022] DELETE /identity/v3/users/33af9a4e3c29426b90dfd5abebde5ed4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:39:03.995838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:39:04.006845 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92536bb-67f4-43a7-b2b6-75b2303f8e2f service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:04.064881 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:04.065497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:04.065768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:04.066035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] PATH_INFO: `/v3/projects/a946788afc074f079618f1a848ec0f8c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:04.067149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a946788afc074f079618f1a848ec0f8c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:39:04.069056 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:39:04.171236 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a946788afc074f079618f1a848ec0f8c', 'request_id': 'req-fa482eb8-5f17-4317-9e8b-b89dab1891e1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.176768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a946788afc074f079618f1a848ec0f8c', 'request_id': 'req-fa482eb8-5f17-4317-9e8b-b89dab1891e1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.182979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a946788afc074f079618f1a848ec0f8c', 'request_id': 'req-fa482eb8-5f17-4317-9e8b-b89dab1891e1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.232863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] The token cache is being invalidate because project a946788afc074f079618f1a848ec0f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:39:04.233150 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.233770 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.235238 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1221/2457] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:39:03 2022] DELETE /identity/v3/projects/a946788afc074f079618f1a848ec0f8c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:39:04.242651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:39:04.242651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:04.253634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb029b-d2a1-4f8f-8247-06ffcb314f39 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:04.310657 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:04.311273 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:04.311549 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:04.311818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] PATH_INFO: `/v3/users/af8e9a0d60b347229e024eedc35cd8e1` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:04.312945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8e9a0d60b347229e024eedc35cd8e1)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:39:04.314837 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:39:04.320027 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:39:04.459572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8e9a0d60b347229e024eedc35cd8e1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.460233 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8e9a0d60b347229e024eedc35cd8e1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.471973 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8e9a0d60b347229e024eedc35cd8e1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.479522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8e9a0d60b347229e024eedc35cd8e1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.480153 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-f5f8f3a4-40d0-47ce-bb7a-8db9767627da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8e9a0d60b347229e024eedc35cd8e1'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.530866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1237/2458] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:39:04 2022] DELETE /identity/v3/users/af8e9a0d60b347229e024eedc35cd8e1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:39:04.767231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:39:04.768784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:39:04.783141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa482eb8-5f17-4317-9e8b-b89dab1891e1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:39:04.800825 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:39:04.847073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:39:04.851088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:39:04.851088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:39:04.851088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] PATH_INFO: `/v3/projects/b565cd9516694f6d80be1675d6b071d0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:39:04.851088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b565cd9516694f6d80be1675d6b071d0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:39:04.851785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:39:04.949450 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b565cd9516694f6d80be1675d6b071d0', 'request_id': 'req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.954891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b565cd9516694f6d80be1675d6b071d0', 'request_id': 'req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:04.962043 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b565cd9516694f6d80be1675d6b071d0', 'request_id': 'req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:05.027587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] The token cache is being invalidate because project b565cd9516694f6d80be1675d6b071d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:39:05.028203 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:05.028506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:39:05.030006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1222/2459] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:39:04 2022] DELETE /identity/v3/projects/b565cd9516694f6d80be1675d6b071d0 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:39:14.527121 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:39:15.024901 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:40:01.411519 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0778d9-0b30-4e51-930d-a3d342f0d6ae None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:01.412088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0778d9-0b30-4e51-930d-a3d342f0d6ae None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:01.412088 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0778d9-0b30-4e51-930d-a3d342f0d6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:01.430623 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-8e0778d9-0b30-4e51-930d-a3d342f0d6ae None None] MFA Rules not processed for user `54fcced6d8d943608b940d1dab001540`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:40:01.431810 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8e0778d9-0b30-4e51-930d-a3d342f0d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:01.484725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1238/2460] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:40:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:40:01.544015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:01.546058 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:01.604475 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f16e8-4397-46b8-a8f8-b1b7cf64e89e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:01.608575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:01.609170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:01.609443 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:01.609709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:01.611402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:01.670195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:01.671609 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:01.680521 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1223/2461] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:11.478213 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=25 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:40:11.676349 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=48 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:40:13.317724 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56accc9d-eabc-4208-81aa-b4db977cfafa None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:13.318217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56accc9d-eabc-4208-81aa-b4db977cfafa None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:13.318331 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56accc9d-eabc-4208-81aa-b4db977cfafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:13.337648 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-56accc9d-eabc-4208-81aa-b4db977cfafa None None] MFA Rules not processed for user `cbdf62fbe49d44a58c294ac2c8b22ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:40:13.338824 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-56accc9d-eabc-4208-81aa-b4db977cfafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:13.345604 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1239/2462] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:40:16.484048 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:16.485528 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:16.541821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48d866-46e3-4b7b-b637-879e4055794a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:16.545376 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:16.546516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:16.546794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:16.547066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:16.548777 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:16.618180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:16.619525 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:16.630373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1224/2463] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:21.300061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56accc9d-eabc-4208-81aa-b4db977cfafa None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:21.301545 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-56accc9d-eabc-4208-81aa-b4db977cfafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:21.314013 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56accc9d-eabc-4208-81aa-b4db977cfafa None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:21.317481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:21.318061 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:21.318335 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:21.318602 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:21.320323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:21.333482 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:21.334873 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:21.345420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1240/2464] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:26.625559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=54 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:40:31.340751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=9 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:40:33.398282 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:40:33.407352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9786e0-6c3f-48ed-b47f-0535e093062e service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:40:33.410878 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:40:33.411467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:40:33.411745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:40:33.412015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:40:33.413907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:40:33.478089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:40:33.479511 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:40:33.489659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1225/2465] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:40:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:40:43.485302 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=30 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:34.598370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:34.614286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:34.614286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb19a2c-66e7-45c1-a6d7-7ca0b2df703a service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:34.619587 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2b7ff-ed27-46f8-b9f1-5ce4ed204565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:34.620183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b7ff-ed27-46f8-b9f1-5ce4ed204565 service nova] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:34.620493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b7ff-ed27-46f8-b9f1-5ce4ed204565 service nova] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:34.620761 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2b7ff-ed27-46f8-b9f1-5ce4ed204565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:34.622486 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-95b2b7ff-ed27-46f8-b9f1-5ce4ed204565 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:34.633704 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b7ff-ed27-46f8-b9f1-5ce4ed204565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:34.635283 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2b7ff-ed27-46f8-b9f1-5ce4ed204565 service nova] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:34.644375 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1241/2466] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:44.640179 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:41:51.469325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:51.470890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:51.487252 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbd47d-433c-46dd-8880-af5e9d3f095c service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:51.493301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:51.493927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:51.494207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:51.494478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:51.496434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:51.511971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:51.513840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:51.529773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1226/2467] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:56.095947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d6e3-b629-4193-8f4e-376d34164b3d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:56.096414 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d6e3-b629-4193-8f4e-376d34164b3d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:56.096528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6d6e3-b629-4193-8f4e-376d34164b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:56.115981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-1ec6d6e3-b629-4193-8f4e-376d34164b3d None None] MFA Rules not processed for user `54fcced6d8d943608b940d1dab001540`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:41:56.117208 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1ec6d6e3-b629-4193-8f4e-376d34164b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:56.123970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1242/2468] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:41:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:41:56.794355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:56.795810 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:56.868969 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8a19a-6278-49b3-8f37-26338dd382fd service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:56.873823 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:56.874420 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:56.874721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:56.874990 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:56.876767 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:56.887782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:56.889366 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:56.899629 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1227/2469] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:41:59.513240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6d6e3-b629-4193-8f4e-376d34164b3d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:41:59.524175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6d6e3-b629-4193-8f4e-376d34164b3d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:41:59.527705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:41:59.528322 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:41:59.531950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:41:59.531950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:41:59.531950 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:41:59.540596 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:41:59.542002 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:41:59.550222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1243/2470] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:41:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:06.894831 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=38 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:08.198217 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:08.210844 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a524660-f9dd-441b-b7eb-75eea9349e6a service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:08.216290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5b4b17788944f8a0991cb62b2a7205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:08.220321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:08.220321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:08.220321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:08.220321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:08.231701 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:08.233277 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:08.245634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1228/2471] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:09.546503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=9 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:18.239356 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:40.969364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:40.978269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc521fb0-4b18-4d14-b10e-4d82f97f46c4 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:40.982465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:40.984396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:40.984396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:40.984396 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:40.985415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:41.049424 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:41.050840 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:41.060890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1244/2472] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:44.426508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:44.437835 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128c0ff-c0e3-4a96-a90b-c5bb5bad3bf5 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:44.442712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:44.443300 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:44.443576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:44.443849 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:44.445822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:44.460542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:44.477382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:44.477382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1229/2473] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:42:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:42:47.881884 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:47.883404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:47.896619 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1f3ce-92a0-49d1-943e-4a9bcd394eb8 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:47.901284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:47.901933 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:47.902210 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:47.902981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] PATH_INFO: `/v3/users/cbdf62fbe49d44a58c294ac2c8b22ef5` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:47.904112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbdf62fbe49d44a58c294ac2c8b22ef5)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:47.906042 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:48.083379 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbdf62fbe49d44a58c294ac2c8b22ef5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.084090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbdf62fbe49d44a58c294ac2c8b22ef5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.097669 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbdf62fbe49d44a58c294ac2c8b22ef5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.104538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbdf62fbe49d44a58c294ac2c8b22ef5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.105211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbdf62fbe49d44a58c294ac2c8b22ef5'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.171564 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1245/2474] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:47 2022] DELETE /identity/v3/users/cbdf62fbe49d44a58c294ac2c8b22ef5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:48.374245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:48.387466 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58152f1d-0fb4-406d-be44-ca3a5b991574 service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:48.445545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:48.446098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:48.446379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:48.446653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] PATH_INFO: `/v3/projects/46d41e46b7e3415684c32b22f31eefb5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:48.447769 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d41e46b7e3415684c32b22f31eefb5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:48.450340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:48.590484 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d41e46b7e3415684c32b22f31eefb5', 'request_id': 'req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.596417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d41e46b7e3415684c32b22f31eefb5', 'request_id': 'req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.603106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d41e46b7e3415684c32b22f31eefb5', 'request_id': 'req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.703015 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] The token cache is being invalidate because project 46d41e46b7e3415684c32b22f31eefb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:48.703258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.703877 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.705412 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1230/2475] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:48 2022] DELETE /identity/v3/projects/46d41e46b7e3415684c32b22f31eefb5 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:48.709543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:48.710932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:48.724211 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88f382-39f0-4054-a839-680db5d721ee admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:48.778917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:48.779521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:48.779796 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:48.780058 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] PATH_INFO: `/v3/users/4c42ea650d2d4aa3bc2838c3bb16a18c` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:48.781192 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c42ea650d2d4aa3bc2838c3bb16a18c)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:48.783073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:48.788187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:48.968637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c42ea650d2d4aa3bc2838c3bb16a18c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.968637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c42ea650d2d4aa3bc2838c3bb16a18c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.976870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c42ea650d2d4aa3bc2838c3bb16a18c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.983725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c42ea650d2d4aa3bc2838c3bb16a18c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:48.984386 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c42ea650d2d4aa3bc2838c3bb16a18c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:49.034898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1246/2476] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:42:48 2022] DELETE /identity/v3/users/4c42ea650d2d4aa3bc2838c3bb16a18c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:49.321467 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:42:49.322870 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:42:49.336536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a42ca0-4b9c-44e9-a61a-e84cbd8b0273 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:42:49.392228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:42:49.392904 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:42:49.393138 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:42:49.393405 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] PATH_INFO: `/v3/projects/0fd9e67249d845a8b4d45bcf1dbb14fc` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:42:49.394498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd9e67249d845a8b4d45bcf1dbb14fc)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:42:49.396400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:42:49.416321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:49.491961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd9e67249d845a8b4d45bcf1dbb14fc', 'request_id': 'req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:49.497873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd9e67249d845a8b4d45bcf1dbb14fc', 'request_id': 'req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:49.503920 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd9e67249d845a8b4d45bcf1dbb14fc', 'request_id': 'req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:49.571185 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] The token cache is being invalidate because project 0fd9e67249d845a8b4d45bcf1dbb14fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:42:49.571430 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:49.572036 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:42:49.573513 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1231/2477] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:42:49 2022] DELETE /identity/v3/projects/0fd9e67249d845a8b4d45bcf1dbb14fc => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:42:59.031257 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:42:59.567706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:40.038351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:40.039768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:40.182064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45965a89-5a7a-4878-b078-00877d78d76a admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:40.241750 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:40.242351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:40.242748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:40.243022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:40.244757 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:40.256461 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:40.258041 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:40.267936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1247/2478] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:48.021394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:48.022786 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:48.081932 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63ede6-1007-4a53-af45-1f4cf5d1cfd8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:48.085564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:48.086162 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:48.086440 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:48.086713 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:48.088439 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:48.151638 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:48.153083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:48.163083 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1232/2479] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:43:50.263866 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:51.520340 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:51.521898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:51.537670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dde29-fc57-4fdd-add6-43645900a979 service placement] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:51.542477 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:51.544779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:51.544779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:51.544779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] PATH_INFO: `/v3/users/54fcced6d8d943608b940d1dab001540` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:51.545080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] RBAC: Authorizing `identity:delete_user(user_id=54fcced6d8d943608b940d1dab001540)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:51.546854 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:51.726682 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54fcced6d8d943608b940d1dab001540'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:51.727341 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54fcced6d8d943608b940d1dab001540'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:51.739014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54fcced6d8d943608b940d1dab001540'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:51.745862 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54fcced6d8d943608b940d1dab001540'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:51.746481 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-772f7355-58ad-4e51-93db-a9dc1cc5d60f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54fcced6d8d943608b940d1dab001540'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:51.816186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1248/2480] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:43:51 2022] DELETE /identity/v3/users/54fcced6d8d943608b940d1dab001540 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:52.046779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:52.057330 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3e63d-2936-45ef-b243-558cb1e6a059 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:52.116299 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:52.116916 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.117194 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.117460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] PATH_INFO: `/v3/projects/0a7896472f034eb4b3b028d528cfbac0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.118557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7896472f034eb4b3b028d528cfbac0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:52.120465 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:52.226131 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:52.227505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7896472f034eb4b3b028d528cfbac0', 'request_id': 'req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:52.232890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7896472f034eb4b3b028d528cfbac0', 'request_id': 'req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:52.239181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7896472f034eb4b3b028d528cfbac0', 'request_id': 'req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:52.319261 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] The token cache is being invalidate because project 0a7896472f034eb4b3b028d528cfbac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:43:52.319487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:52.320102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec30a6bc-492b-4f03-8a0e-1f484db5dd68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:43:52.321596 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1233/2481] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:43:52 2022] DELETE /identity/v3/projects/0a7896472f034eb4b3b028d528cfbac0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:52.326475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3adc-b183-4f13-bfaa-4d824e270932 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.326762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3adc-b183-4f13-bfaa-4d824e270932 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.327035 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b3adc-b183-4f13-bfaa-4d824e270932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.346352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c86b3adc-b183-4f13-bfaa-4d824e270932 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:52.347543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c86b3adc-b183-4f13-bfaa-4d824e270932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:52.404900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1249/2482] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:52.412986 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b6eb8-31e3-450e-8dc1-da1a4779b533 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.413297 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b6eb8-31e3-450e-8dc1-da1a4779b533 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.413576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b6eb8-31e3-450e-8dc1-da1a4779b533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.433756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-117b6eb8-31e3-450e-8dc1-da1a4779b533 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:52.434945 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-117b6eb8-31e3-450e-8dc1-da1a4779b533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:52.443363 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1234/2483] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:52.447630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b3adc-b183-4f13-bfaa-4d824e270932 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:52.456942 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b3adc-b183-4f13-bfaa-4d824e270932 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:52.461368 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:52.461948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.462222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.462489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.463116 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:43:52.463415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:52.465023 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:52.476141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1250/2484] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:43:52.480195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b6eb8-31e3-450e-8dc1-da1a4779b533 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:52.489868 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b6eb8-31e3-450e-8dc1-da1a4779b533 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:52.494589 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc868b-e5c2-4f9e-8832-fc997a1de87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:52.495170 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc868b-e5c2-4f9e-8832-fc997a1de87b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.498535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc868b-e5c2-4f9e-8832-fc997a1de87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.498535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc868b-e5c2-4f9e-8832-fc997a1de87b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.498535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc868b-e5c2-4f9e-8832-fc997a1de87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:52.498535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc868b-e5c2-4f9e-8832-fc997a1de87b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:52.611563 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1235/2485] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:52 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:52.616095 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:52.625898 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f619d-cebe-473e-b192-9ce7aa0e8517 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:52.680974 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f295e-8726-4764-ad5e-a74681561687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:52.681558 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f295e-8726-4764-ad5e-a74681561687 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.681831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f295e-8726-4764-ad5e-a74681561687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.682098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f295e-8726-4764-ad5e-a74681561687 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.682725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f295e-8726-4764-ad5e-a74681561687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:52.684651 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f295e-8726-4764-ad5e-a74681561687 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:52.692450 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:52.775907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1251/2486] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:52 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:52.779873 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc868b-e5c2-4f9e-8832-fc997a1de87b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:52.790853 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc868b-e5c2-4f9e-8832-fc997a1de87b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:52.795360 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3854a-fcf6-4b5e-b3bb-b46970d7d3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:52.795942 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3854a-fcf6-4b5e-b3bb-b46970d7d3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.796216 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3854a-fcf6-4b5e-b3bb-b46970d7d3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.796515 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3854a-fcf6-4b5e-b3bb-b46970d7d3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.797139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3854a-fcf6-4b5e-b3bb-b46970d7d3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:52.799006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3854a-fcf6-4b5e-b3bb-b46970d7d3f2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:52.809632 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1236/2487] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:52.813634 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f295e-8726-4764-ad5e-a74681561687 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:52.823792 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f295e-8726-4764-ad5e-a74681561687 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:52.828503 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced65c37-ecb4-4e69-a450-812efbedfdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:52.829106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65c37-ecb4-4e69-a450-812efbedfdec admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.829607 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65c37-ecb4-4e69-a450-812efbedfdec admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.829882 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced65c37-ecb4-4e69-a450-812efbedfdec admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.830505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced65c37-ecb4-4e69-a450-812efbedfdec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:52.832161 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced65c37-ecb4-4e69-a450-812efbedfdec admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:52.843021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1252/2488] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:52.847535 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3854a-fcf6-4b5e-b3bb-b46970d7d3f2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:52.857245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3854a-fcf6-4b5e-b3bb-b46970d7d3f2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:52.861848 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9c85c-6413-4d50-b718-634653262efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:52.863118 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9c85c-6413-4d50-b718-634653262efc admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.863394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9c85c-6413-4d50-b718-634653262efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.864253 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9c85c-6413-4d50-b718-634653262efc admin admin] PATH_INFO: `/v3/projects/c52cd9d2986941a486d38f077f13bb81/users/f0724135ec4e47989a0a86a56d7c9eb7/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.878456 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9c85c-6413-4d50-b718-634653262efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52cd9d2986941a486d38f077f13bb81, user_id=f0724135ec4e47989a0a86a56d7c9eb7, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:52.881153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9c85c-6413-4d50-b718-634653262efc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:52.954403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1237/2489] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:52 2022] PUT /identity/v3/projects/c52cd9d2986941a486d38f077f13bb81/users/f0724135ec4e47989a0a86a56d7c9eb7/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:52.965452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5772e47-c310-4fd9-8d42-210b5c872068 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:52.965762 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5772e47-c310-4fd9-8d42-210b5c872068 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:52.966037 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5772e47-c310-4fd9-8d42-210b5c872068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:52.993354 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-a5772e47-c310-4fd9-8d42-210b5c872068 None None] MFA Rules not processed for user `f0724135ec4e47989a0a86a56d7c9eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:52.994552 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-a5772e47-c310-4fd9-8d42-210b5c872068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:53.057865 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1253/2490] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:53.062385 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9c85c-6413-4d50-b718-634653262efc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:53.124201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9c85c-6413-4d50-b718-634653262efc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:53.129400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b37af-2b7f-4496-9668-de6e4e702a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:53.129979 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b37af-2b7f-4496-9668-de6e4e702a6c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:53.130258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b37af-2b7f-4496-9668-de6e4e702a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:53.130531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b37af-2b7f-4496-9668-de6e4e702a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:53.131152 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b37af-2b7f-4496-9668-de6e4e702a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:53.133066 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b37af-2b7f-4496-9668-de6e4e702a6c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:53.216995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1238/2491] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:43:53 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:43:53.221106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5772e47-c310-4fd9-8d42-210b5c872068 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:53.231606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5772e47-c310-4fd9-8d42-210b5c872068 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:53.289016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb54d56-f3ec-4d6b-ae18-5a0411cc389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:53.289671 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb54d56-f3ec-4d6b-ae18-5a0411cc389d admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:53.289951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb54d56-f3ec-4d6b-ae18-5a0411cc389d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:53.290222 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb54d56-f3ec-4d6b-ae18-5a0411cc389d admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:53.290859 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb54d56-f3ec-4d6b-ae18-5a0411cc389d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:53.292793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb54d56-f3ec-4d6b-ae18-5a0411cc389d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:53.371205 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1254/2492] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:43:53 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:43:53.375172 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b37af-2b7f-4496-9668-de6e4e702a6c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:53.385222 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b37af-2b7f-4496-9668-de6e4e702a6c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:53.389845 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272e7de-2e45-4312-b693-6bc0d5767255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:53.390458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272e7de-2e45-4312-b693-6bc0d5767255 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:53.390738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272e7de-2e45-4312-b693-6bc0d5767255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:53.391009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272e7de-2e45-4312-b693-6bc0d5767255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:53.391639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272e7de-2e45-4312-b693-6bc0d5767255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:53.393240 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272e7de-2e45-4312-b693-6bc0d5767255 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:53.403587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1239/2493] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:53.411489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb54d56-f3ec-4d6b-ae18-5a0411cc389d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:53.421420 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb54d56-f3ec-4d6b-ae18-5a0411cc389d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:53.426039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cc1e2-d3c2-43b8-aba3-d9dd8502e474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:53.427220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc1e2-d3c2-43b8-aba3-d9dd8502e474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:53.427499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc1e2-d3c2-43b8-aba3-d9dd8502e474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:53.427772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cc1e2-d3c2-43b8-aba3-d9dd8502e474 admin admin] PATH_INFO: `/v3/projects/c20b2d5ca63c4271a3cab682450fea39/users/e40fb20e914e46508ba2b95df76f7b23/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:53.442197 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cc1e2-d3c2-43b8-aba3-d9dd8502e474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20b2d5ca63c4271a3cab682450fea39, user_id=e40fb20e914e46508ba2b95df76f7b23, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:53.445145 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cc1e2-d3c2-43b8-aba3-d9dd8502e474 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:53.524709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1255/2494] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:53 2022] PUT /identity/v3/projects/c20b2d5ca63c4271a3cab682450fea39/users/e40fb20e914e46508ba2b95df76f7b23/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:53.528741 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272e7de-2e45-4312-b693-6bc0d5767255 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:53.539361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272e7de-2e45-4312-b693-6bc0d5767255 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:53.584346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:53.596683 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79867e-b377-4f01-83e7-d8702530c91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:53.597284 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79867e-b377-4f01-83e7-d8702530c91c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:53.597565 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79867e-b377-4f01-83e7-d8702530c91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:53.598326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79867e-b377-4f01-83e7-d8702530c91c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:53.598966 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79867e-b377-4f01-83e7-d8702530c91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:53.600559 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79867e-b377-4f01-83e7-d8702530c91c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:53.610343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1240/2495] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:43:53.614817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cc1e2-d3c2-43b8-aba3-d9dd8502e474 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:53.625526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cc1e2-d3c2-43b8-aba3-d9dd8502e474 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:53.630228 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:53.630876 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:53.631158 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:53.631436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] PATH_INFO: `/v3/projects/c20b2d5ca63c4271a3cab682450fea39/users/e40fb20e914e46508ba2b95df76f7b23/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:53.633447 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20b2d5ca63c4271a3cab682450fea39, user_id=e40fb20e914e46508ba2b95df76f7b23, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:53.636073 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:53.646615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1256/2496] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:43:53 2022] PUT /identity/v3/projects/c20b2d5ca63c4271a3cab682450fea39/users/e40fb20e914e46508ba2b95df76f7b23/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:43:53.658122 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a7eb-d313-4760-ad65-af1149fe8605 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:53.658434 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a7eb-d313-4760-ad65-af1149fe8605 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:53.658709 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909a7eb-d313-4760-ad65-af1149fe8605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:53.687106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b909a7eb-d313-4760-ad65-af1149fe8605 None None] MFA Rules not processed for user `e40fb20e914e46508ba2b95df76f7b23`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:43:53.688338 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b909a7eb-d313-4760-ad65-af1149fe8605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:53.755325 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1241/2497] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:43:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:43:53.766646 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:53.768357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:43:53.791577 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:43:53.827509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806df956-75ec-4934-a9ee-93571f4ef61d admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:53.831182 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed427-1922-4358-862b-cc1ba384b4c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:53.831772 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed427-1922-4358-862b-cc1ba384b4c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:53.832050 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed427-1922-4358-862b-cc1ba384b4c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:53.832883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed427-1922-4358-862b-cc1ba384b4c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:53.841806 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed427-1922-4358-862b-cc1ba384b4c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:53.843220 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed427-1922-4358-862b-cc1ba384b4c0 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:53.853425 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1257/2498] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:43:54.041968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909a7eb-d313-4760-ad65-af1149fe8605 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:43:54.051616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909a7eb-d313-4760-ad65-af1149fe8605 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:43:54.055186 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b73200-f327-456c-9bb8-617239121d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:43:54.055783 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b73200-f327-456c-9bb8-617239121d5a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:43:54.056068 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b73200-f327-456c-9bb8-617239121d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:43:54.056369 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b73200-f327-456c-9bb8-617239121d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:43:54.113562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b73200-f327-456c-9bb8-617239121d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:43:54.114999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b73200-f327-456c-9bb8-617239121d5a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:43:54.123735 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1242/2499] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:43:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:03.852237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=17 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:04.119732 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=70 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:05.597812 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed427-1922-4358-862b-cc1ba384b4c0 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:05.679364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed427-1922-4358-862b-cc1ba384b4c0 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:05.683871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:05.684526 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:05.684805 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:05.685075 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] PATH_INFO: `/v3/users/f0724135ec4e47989a0a86a56d7c9eb7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:05.686167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0724135ec4e47989a0a86a56d7c9eb7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:05.688071 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:05.823296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0724135ec4e47989a0a86a56d7c9eb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:05.823945 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0724135ec4e47989a0a86a56d7c9eb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:05.836207 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0724135ec4e47989a0a86a56d7c9eb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:05.843267 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0724135ec4e47989a0a86a56d7c9eb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:05.843995 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0724135ec4e47989a0a86a56d7c9eb7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:05.918828 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1258/2500] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:05 2022] DELETE /identity/v3/users/f0724135ec4e47989a0a86a56d7c9eb7 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:05.928476 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323fe0-8af7-4333-bccf-3881b2d51dad None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:05.928949 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323fe0-8af7-4333-bccf-3881b2d51dad None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:05.929075 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323fe0-8af7-4333-bccf-3881b2d51dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:05.959544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c6323fe0-8af7-4333-bccf-3881b2d51dad None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:05.961368 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c6323fe0-8af7-4333-bccf-3881b2d51dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:06.018692 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1243/2501] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:06.029238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:06.030610 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:06.097689 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb89ca-55b6-4764-915e-02dbd8ec0c6c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:06.101338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063b6d2-4893-4676-8ef9-ee8eda62e181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:06.101938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063b6d2-4893-4676-8ef9-ee8eda62e181 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:06.102217 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063b6d2-4893-4676-8ef9-ee8eda62e181 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:06.102489 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063b6d2-4893-4676-8ef9-ee8eda62e181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:06.109445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:06.111684 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063b6d2-4893-4676-8ef9-ee8eda62e181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:06.113206 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063b6d2-4893-4676-8ef9-ee8eda62e181 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:06.122658 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1259/2502] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:06.526167 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323fe0-8af7-4333-bccf-3881b2d51dad None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:06.527571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c6323fe0-8af7-4333-bccf-3881b2d51dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:06.539432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323fe0-8af7-4333-bccf-3881b2d51dad None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:06.544059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:06.544696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:06.544971 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:06.545239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] PATH_INFO: `/v3/projects/c52cd9d2986941a486d38f077f13bb81` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:06.546568 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52cd9d2986941a486d38f077f13bb81)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:06.548487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:06.656562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52cd9d2986941a486d38f077f13bb81', 'request_id': 'req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:06.662109 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52cd9d2986941a486d38f077f13bb81', 'request_id': 'req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:06.671360 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52cd9d2986941a486d38f077f13bb81', 'request_id': 'req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:06.729422 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] The token cache is being invalidate because project c52cd9d2986941a486d38f077f13bb81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:06.729738 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:06.730353 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:06.731832 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1244/2503] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:06 2022] DELETE /identity/v3/projects/c52cd9d2986941a486d38f077f13bb81 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:06.735802 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063b6d2-4893-4676-8ef9-ee8eda62e181 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:06.747793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063b6d2-4893-4676-8ef9-ee8eda62e181 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:06.802528 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:06.803150 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:06.803431 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:06.804293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] PATH_INFO: `/v3/users/e40fb20e914e46508ba2b95df76f7b23` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:06.805418 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e40fb20e914e46508ba2b95df76f7b23)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:06.807339 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:07.028871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e40fb20e914e46508ba2b95df76f7b23'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.029454 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e40fb20e914e46508ba2b95df76f7b23'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.041946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e40fb20e914e46508ba2b95df76f7b23'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.048811 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e40fb20e914e46508ba2b95df76f7b23'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.049440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-906b0b5d-3205-47da-99bd-0e49028f7d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e40fb20e914e46508ba2b95df76f7b23'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.123644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1260/2504] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:06 2022] DELETE /identity/v3/users/e40fb20e914e46508ba2b95df76f7b23 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:07.544660 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:07.545228 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:07.558248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9eecf-0f7b-4ff2-b06c-1cb043eade4f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:07.615785 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:07.616435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:07.616717 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:07.616988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] PATH_INFO: `/v3/projects/c20b2d5ca63c4271a3cab682450fea39` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:07.618085 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20b2d5ca63c4271a3cab682450fea39)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:07.619961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:07.635645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:07.741644 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20b2d5ca63c4271a3cab682450fea39', 'request_id': 'req-d49b9546-8b98-4425-826e-181ef6b96297'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.747072 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20b2d5ca63c4271a3cab682450fea39', 'request_id': 'req-d49b9546-8b98-4425-826e-181ef6b96297'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.753257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20b2d5ca63c4271a3cab682450fea39', 'request_id': 'req-d49b9546-8b98-4425-826e-181ef6b96297'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.803671 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] The token cache is being invalidate because project c20b2d5ca63c4271a3cab682450fea39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:07.803884 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.804653 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d49b9546-8b98-4425-826e-181ef6b96297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:07.806119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1245/2505] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:07 2022] DELETE /identity/v3/projects/c20b2d5ca63c4271a3cab682450fea39 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:07.810962 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556337-7f70-4692-9fb4-7f922728d033 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:07.811448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556337-7f70-4692-9fb4-7f922728d033 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:07.811556 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556337-7f70-4692-9fb4-7f922728d033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:07.831931 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-c6556337-7f70-4692-9fb4-7f922728d033 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:07.833160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-c6556337-7f70-4692-9fb4-7f922728d033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:07.891916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1261/2506] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:07.900379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b5bf0-1ae2-4546-ba41-bb8417a56799 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:07.900666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b5bf0-1ae2-4546-ba41-bb8417a56799 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:07.900940 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b5bf0-1ae2-4546-ba41-bb8417a56799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:07.920190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-407b5bf0-1ae2-4546-ba41-bb8417a56799 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:07.921403 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-407b5bf0-1ae2-4546-ba41-bb8417a56799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:07.929600 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1246/2507] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:07.933629 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556337-7f70-4692-9fb4-7f922728d033 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:07.942951 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556337-7f70-4692-9fb4-7f922728d033 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:07.948294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:07.948893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:07.949177 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:07.949445 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:07.950077 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:44:07.950378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:07.951946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:07.963330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1262/2508] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:44:07.967214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b5bf0-1ae2-4546-ba41-bb8417a56799 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:07.976107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b5bf0-1ae2-4546-ba41-bb8417a56799 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:07.980686 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dc9c6-bd87-4174-9680-0f4488d40965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:07.981258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dc9c6-bd87-4174-9680-0f4488d40965 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:07.981531 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dc9c6-bd87-4174-9680-0f4488d40965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:07.981796 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dc9c6-bd87-4174-9680-0f4488d40965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:07.982404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dc9c6-bd87-4174-9680-0f4488d40965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:07.984257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dc9c6-bd87-4174-9680-0f4488d40965 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.051317 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1247/2509] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:07 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:08.056448 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.061060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:08.066237 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cd819-7609-4088-89fe-e00366570059 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.122936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cf1e3-1c13-4a60-9546-6843c57831eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:08.123529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cf1e3-1c13-4a60-9546-6843c57831eb admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.123808 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cf1e3-1c13-4a60-9546-6843c57831eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.124080 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cf1e3-1c13-4a60-9546-6843c57831eb admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.124723 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cf1e3-1c13-4a60-9546-6843c57831eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:08.126621 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cf1e3-1c13-4a60-9546-6843c57831eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.181804 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1263/2510] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:08 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:44:08.186000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dc9c6-bd87-4174-9680-0f4488d40965 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.196680 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dc9c6-bd87-4174-9680-0f4488d40965 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.201264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc134dc-621c-4b35-9647-17f6e968aa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:08.201900 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc134dc-621c-4b35-9647-17f6e968aa16 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.202180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc134dc-621c-4b35-9647-17f6e968aa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.202482 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc134dc-621c-4b35-9647-17f6e968aa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.203113 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc134dc-621c-4b35-9647-17f6e968aa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:08.204723 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc134dc-621c-4b35-9647-17f6e968aa16 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.215155 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1248/2511] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:08.218900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cf1e3-1c13-4a60-9546-6843c57831eb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.229202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cf1e3-1c13-4a60-9546-6843c57831eb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.233705 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db27884-ed81-4f0e-aafd-74c5b7766e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:08.234278 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db27884-ed81-4f0e-aafd-74c5b7766e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.234554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db27884-ed81-4f0e-aafd-74c5b7766e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.234821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db27884-ed81-4f0e-aafd-74c5b7766e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.235442 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db27884-ed81-4f0e-aafd-74c5b7766e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:08.237038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db27884-ed81-4f0e-aafd-74c5b7766e1f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.247487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1264/2512] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:08.251321 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc134dc-621c-4b35-9647-17f6e968aa16 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.261708 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc134dc-621c-4b35-9647-17f6e968aa16 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.266394 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1f0da-5f6b-46c8-85b9-5f4349514b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:08.267042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f0da-5f6b-46c8-85b9-5f4349514b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.267319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f0da-5f6b-46c8-85b9-5f4349514b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.267612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1f0da-5f6b-46c8-85b9-5f4349514b1d admin admin] PATH_INFO: `/v3/projects/972427d301a94bf9add6283d6a5960a9/users/04c3b31839e54125aaa3c4b308d5b749/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.283326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1f0da-5f6b-46c8-85b9-5f4349514b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=972427d301a94bf9add6283d6a5960a9, user_id=04c3b31839e54125aaa3c4b308d5b749, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:08.285994 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1f0da-5f6b-46c8-85b9-5f4349514b1d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.347310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1249/2513] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:08 2022] PUT /identity/v3/projects/972427d301a94bf9add6283d6a5960a9/users/04c3b31839e54125aaa3c4b308d5b749/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:44:08.363857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486e15-f579-49de-8882-3ae9a76ca6da None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.364136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486e15-f579-49de-8882-3ae9a76ca6da None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.364468 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5486e15-f579-49de-8882-3ae9a76ca6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.391827 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d5486e15-f579-49de-8882-3ae9a76ca6da None None] MFA Rules not processed for user `04c3b31839e54125aaa3c4b308d5b749`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:08.393039 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d5486e15-f579-49de-8882-3ae9a76ca6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:08.454176 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1265/2514] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:44:08.458555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1f0da-5f6b-46c8-85b9-5f4349514b1d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.520784 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1f0da-5f6b-46c8-85b9-5f4349514b1d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.525373 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a11b9-f0be-4341-8ee8-5c087c8c0029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:08.525992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a11b9-f0be-4341-8ee8-5c087c8c0029 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.526266 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a11b9-f0be-4341-8ee8-5c087c8c0029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.526533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a11b9-f0be-4341-8ee8-5c087c8c0029 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.527153 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a11b9-f0be-4341-8ee8-5c087c8c0029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:08.529090 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a11b9-f0be-4341-8ee8-5c087c8c0029 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.600247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1250/2515] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:08 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:08.604943 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5486e15-f579-49de-8882-3ae9a76ca6da None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.615540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5486e15-f579-49de-8882-3ae9a76ca6da None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.671641 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73199f-d56f-45c2-b1ca-133a31332737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:08.672235 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73199f-d56f-45c2-b1ca-133a31332737 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.672543 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73199f-d56f-45c2-b1ca-133a31332737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.672813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73199f-d56f-45c2-b1ca-133a31332737 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.673436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73199f-d56f-45c2-b1ca-133a31332737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:08.675323 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73199f-d56f-45c2-b1ca-133a31332737 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.754408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:08.759801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1266/2516] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:08 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:08.763968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a11b9-f0be-4341-8ee8-5c087c8c0029 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.774386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a11b9-f0be-4341-8ee8-5c087c8c0029 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.779343 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024da7d7-3622-437d-87aa-fc7643cc1612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:08.779981 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024da7d7-3622-437d-87aa-fc7643cc1612 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.780257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024da7d7-3622-437d-87aa-fc7643cc1612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.780619 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024da7d7-3622-437d-87aa-fc7643cc1612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.781249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024da7d7-3622-437d-87aa-fc7643cc1612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:08.782854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024da7d7-3622-437d-87aa-fc7643cc1612 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.793517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1251/2517] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:44:08.797729 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73199f-d56f-45c2-b1ca-133a31332737 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.808008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73199f-d56f-45c2-b1ca-133a31332737 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.813191 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897c575-af29-4652-8abd-275d9e272c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:08.818706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897c575-af29-4652-8abd-275d9e272c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:08.818706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897c575-af29-4652-8abd-275d9e272c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:08.818706 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897c575-af29-4652-8abd-275d9e272c66 admin admin] PATH_INFO: `/v3/projects/714a09971dde43a18fb6034d09232e49/users/cd115f5a70a8460192a5356487a550e9/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:08.829831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897c575-af29-4652-8abd-275d9e272c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714a09971dde43a18fb6034d09232e49, user_id=cd115f5a70a8460192a5356487a550e9, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:08.832570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897c575-af29-4652-8abd-275d9e272c66 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:08.930752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1267/2518] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:08 2022] PUT /identity/v3/projects/714a09971dde43a18fb6034d09232e49/users/cd115f5a70a8460192a5356487a550e9/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:08.934679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024da7d7-3622-437d-87aa-fc7643cc1612 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:08.944797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024da7d7-3622-437d-87aa-fc7643cc1612 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:08.954065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:09.000869 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf12327-b46e-4efa-9aec-e9340bda3bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:09.001460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf12327-b46e-4efa-9aec-e9340bda3bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:09.001739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf12327-b46e-4efa-9aec-e9340bda3bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:09.002011 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf12327-b46e-4efa-9aec-e9340bda3bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:09.002645 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf12327-b46e-4efa-9aec-e9340bda3bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:09.004262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf12327-b46e-4efa-9aec-e9340bda3bb8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:09.014415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1252/2519] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:09.018542 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897c575-af29-4652-8abd-275d9e272c66 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:09.028680 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897c575-af29-4652-8abd-275d9e272c66 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:09.033204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:09.033835 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:09.034186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:09.034464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] PATH_INFO: `/v3/projects/714a09971dde43a18fb6034d09232e49/users/cd115f5a70a8460192a5356487a550e9/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:09.036374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714a09971dde43a18fb6034d09232e49, user_id=cd115f5a70a8460192a5356487a550e9, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:09.039507 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:09.049836 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1268/2520] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:09 2022] PUT /identity/v3/projects/714a09971dde43a18fb6034d09232e49/users/cd115f5a70a8460192a5356487a550e9/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:09.060460 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d9c13-a99b-48e0-8be3-1ee844a4286b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:09.060749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d9c13-a99b-48e0-8be3-1ee844a4286b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:09.061035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d9c13-a99b-48e0-8be3-1ee844a4286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:09.088601 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-675d9c13-a99b-48e0-8be3-1ee844a4286b None None] MFA Rules not processed for user `cd115f5a70a8460192a5356487a550e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:09.089813 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-675d9c13-a99b-48e0-8be3-1ee844a4286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:09.156703 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1253/2521] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:09.169204 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:09.170880 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:09.227823 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609438a-c1be-4bca-992b-8f0e16eaa6f7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:09.231551 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e619c-08e1-463c-95c0-72d070ac3ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:09.232132 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e619c-08e1-463c-95c0-72d070ac3ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:09.232441 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e619c-08e1-463c-95c0-72d070ac3ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:09.232714 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e619c-08e1-463c-95c0-72d070ac3ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:09.241538 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e619c-08e1-463c-95c0-72d070ac3ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:09.242952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e619c-08e1-463c-95c0-72d070ac3ba0 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:09.252168 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1269/2522] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:10.171747 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d9c13-a99b-48e0-8be3-1ee844a4286b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.233765 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d9c13-a99b-48e0-8be3-1ee844a4286b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.238293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.238912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.239190 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.239461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] PATH_INFO: `/v3/users/04c3b31839e54125aaa3c4b308d5b749` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.240587 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c3b31839e54125aaa3c4b308d5b749)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.242551 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.422191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:10.430627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c3b31839e54125aaa3c4b308d5b749'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.437537 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c3b31839e54125aaa3c4b308d5b749'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.449246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c3b31839e54125aaa3c4b308d5b749'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.529972 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c3b31839e54125aaa3c4b308d5b749'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.530953 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c3b31839e54125aaa3c4b308d5b749'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:10.537651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1254/2523] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:10 2022] DELETE /identity/v3/users/04c3b31839e54125aaa3c4b308d5b749 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:10.542255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f6802-ae2e-4fe1-bf24-700a296b9573 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.542730 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f6802-ae2e-4fe1-bf24-700a296b9573 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.542872 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f6802-ae2e-4fe1-bf24-700a296b9573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.561523 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d92f6802-ae2e-4fe1-bf24-700a296b9573 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:10.562824 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d92f6802-ae2e-4fe1-bf24-700a296b9573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:10.620818 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1270/2524] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:44:10.630461 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:10.631847 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:10.688555 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda061ea-d707-4fb2-9ca1-e76aefb50e38 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:10.691995 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5df2c4-b5bc-4299-94d4-c38bcde442ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:10.692612 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5df2c4-b5bc-4299-94d4-c38bcde442ef service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:10.692887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5df2c4-b5bc-4299-94d4-c38bcde442ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:10.693423 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5df2c4-b5bc-4299-94d4-c38bcde442ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:10.702599 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5df2c4-b5bc-4299-94d4-c38bcde442ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:10.703999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5df2c4-b5bc-4299-94d4-c38bcde442ef service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:10.713678 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1255/2525] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:11.093798 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f6802-ae2e-4fe1-bf24-700a296b9573 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:11.095334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d92f6802-ae2e-4fe1-bf24-700a296b9573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:11.108057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f6802-ae2e-4fe1-bf24-700a296b9573 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:11.112822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:11.113457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:11.113743 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:11.114009 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] PATH_INFO: `/v3/projects/972427d301a94bf9add6283d6a5960a9` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:11.115119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=972427d301a94bf9add6283d6a5960a9)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:11.117047 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:11.244849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972427d301a94bf9add6283d6a5960a9', 'request_id': 'req-9dd16415-3905-4267-97b2-e798c99a2a1c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.330225 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972427d301a94bf9add6283d6a5960a9', 'request_id': 'req-9dd16415-3905-4267-97b2-e798c99a2a1c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.337338 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972427d301a94bf9add6283d6a5960a9', 'request_id': 'req-9dd16415-3905-4267-97b2-e798c99a2a1c'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.338555 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:11.348028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] The token cache is being invalidate because project 972427d301a94bf9add6283d6a5960a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:11.348318 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.348946 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.350427 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1271/2526] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:11 2022] DELETE /identity/v3/projects/972427d301a94bf9add6283d6a5960a9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:11.354545 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5df2c4-b5bc-4299-94d4-c38bcde442ef service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:11.364379 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5df2c4-b5bc-4299-94d4-c38bcde442ef service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:11.419760 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:11.420438 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:11.420719 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:11.420991 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] PATH_INFO: `/v3/users/cd115f5a70a8460192a5356487a550e9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:11.422102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd115f5a70a8460192a5356487a550e9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:11.424000 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:11.578670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd115f5a70a8460192a5356487a550e9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.586176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd115f5a70a8460192a5356487a550e9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.588057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:11.598214 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd115f5a70a8460192a5356487a550e9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.695552 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd115f5a70a8460192a5356487a550e9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.696163 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-05042c06-ff43-4a77-ae2d-cd9a8639cab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd115f5a70a8460192a5356487a550e9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:11.702098 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1256/2527] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:11 2022] DELETE /identity/v3/users/cd115f5a70a8460192a5356487a550e9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:12.109064 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:12.110505 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:12.123832 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd16415-3905-4267-97b2-e798c99a2a1c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:12.180982 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:12.181597 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.181875 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.182165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] PATH_INFO: `/v3/projects/714a09971dde43a18fb6034d09232e49` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.183356 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=714a09971dde43a18fb6034d09232e49)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:12.185266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:12.278436 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714a09971dde43a18fb6034d09232e49', 'request_id': 'req-9bded943-f066-4a90-9de6-fd2f648be292'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:12.336391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714a09971dde43a18fb6034d09232e49', 'request_id': 'req-9bded943-f066-4a90-9de6-fd2f648be292'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:12.342779 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714a09971dde43a18fb6034d09232e49', 'request_id': 'req-9bded943-f066-4a90-9de6-fd2f648be292'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:12.353098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] The token cache is being invalidate because project 714a09971dde43a18fb6034d09232e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:12.353378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:12.353987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9bded943-f066-4a90-9de6-fd2f648be292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:12.355430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1272/2528] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:12 2022] DELETE /identity/v3/projects/714a09971dde43a18fb6034d09232e49 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:12.360504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641db20-5ca5-4b1e-85c9-e474c5fef1be None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.361008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641db20-5ca5-4b1e-85c9-e474c5fef1be None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.361008 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641db20-5ca5-4b1e-85c9-e474c5fef1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.380447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c641db20-5ca5-4b1e-85c9-e474c5fef1be None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:12.381670 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c641db20-5ca5-4b1e-85c9-e474c5fef1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:12.440408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1257/2529] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:12.448883 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37bd8c-c307-4ca4-ba3a-10dc1409418d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.449167 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37bd8c-c307-4ca4-ba3a-10dc1409418d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.449455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37bd8c-c307-4ca4-ba3a-10dc1409418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.468632 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-3f37bd8c-c307-4ca4-ba3a-10dc1409418d None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:12.469814 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-3f37bd8c-c307-4ca4-ba3a-10dc1409418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:12.477376 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1273/2530] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:12.481284 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641db20-5ca5-4b1e-85c9-e474c5fef1be None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:12.490778 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641db20-5ca5-4b1e-85c9-e474c5fef1be None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:12.496598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:12.496598 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.496854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.496999 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.497634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:44:12.497936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:12.499523 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:12.511534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1258/2531] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:44:12.515585 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37bd8c-c307-4ca4-ba3a-10dc1409418d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:12.524616 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37bd8c-c307-4ca4-ba3a-10dc1409418d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:12.529126 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ceedf-86ef-4e5e-878f-85295b16f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:12.531025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ceedf-86ef-4e5e-878f-85295b16f9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.531025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ceedf-86ef-4e5e-878f-85295b16f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.531025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ceedf-86ef-4e5e-878f-85295b16f9f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.531025 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ceedf-86ef-4e5e-878f-85295b16f9f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:12.532709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ceedf-86ef-4e5e-878f-85295b16f9f9 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:12.611537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1274/2532] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:12 2022] POST /identity/v3/projects => generated 357 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:12.615454 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:12.625705 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d1373-2763-4c13-9657-49f8fabbe96e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:12.682247 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5d614-a8ea-425c-9650-33ffbceba07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:12.682840 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d614-a8ea-425c-9650-33ffbceba07f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.683115 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d614-a8ea-425c-9650-33ffbceba07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.683382 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5d614-a8ea-425c-9650-33ffbceba07f admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.684004 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5d614-a8ea-425c-9650-33ffbceba07f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:12.685905 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5d614-a8ea-425c-9650-33ffbceba07f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:12.781380 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1259/2533] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:12 2022] POST /identity/v3/users => generated 279 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:44:12.785533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ceedf-86ef-4e5e-878f-85295b16f9f9 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:12.796057 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ceedf-86ef-4e5e-878f-85295b16f9f9 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:12.800624 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31068a9-96ec-4014-b78a-860b4a9974eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:12.801202 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31068a9-96ec-4014-b78a-860b4a9974eb admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.801475 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31068a9-96ec-4014-b78a-860b4a9974eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.801742 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31068a9-96ec-4014-b78a-860b4a9974eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.802367 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31068a9-96ec-4014-b78a-860b4a9974eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:12.803934 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31068a9-96ec-4014-b78a-860b4a9974eb admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:12.814198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1275/2534] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:12.817907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5d614-a8ea-425c-9650-33ffbceba07f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:12.828123 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5d614-a8ea-425c-9650-33ffbceba07f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:12.832667 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f601d52-c618-4ad6-8fbc-ce6dd2cce63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:12.833245 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601d52-c618-4ad6-8fbc-ce6dd2cce63b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.833522 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601d52-c618-4ad6-8fbc-ce6dd2cce63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.833794 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f601d52-c618-4ad6-8fbc-ce6dd2cce63b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.834414 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f601d52-c618-4ad6-8fbc-ce6dd2cce63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:12.837617 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f601d52-c618-4ad6-8fbc-ce6dd2cce63b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:12.847998 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1260/2535] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:12.852580 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31068a9-96ec-4014-b78a-860b4a9974eb admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:12.862273 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31068a9-96ec-4014-b78a-860b4a9974eb admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:12.867316 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62746-2ab7-461f-80b1-16c5d14d6063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:12.867947 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62746-2ab7-461f-80b1-16c5d14d6063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:12.868224 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62746-2ab7-461f-80b1-16c5d14d6063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:12.868533 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62746-2ab7-461f-80b1-16c5d14d6063 admin admin] PATH_INFO: `/v3/projects/ca89ab47fdb64e67b070a60f9258ca31/users/b5e43c3f566f4647b98e8882dcf293e7/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:12.882656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c62746-2ab7-461f-80b1-16c5d14d6063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca89ab47fdb64e67b070a60f9258ca31, user_id=b5e43c3f566f4647b98e8882dcf293e7, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:12.885444 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c62746-2ab7-461f-80b1-16c5d14d6063 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:12.947698 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1276/2536] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:12 2022] PUT /identity/v3/projects/ca89ab47fdb64e67b070a60f9258ca31/users/b5e43c3f566f4647b98e8882dcf293e7/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:13.011651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e64cb6-540b-4cad-860b-5cfb8436a39f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.011918 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e64cb6-540b-4cad-860b-5cfb8436a39f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.012192 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e64cb6-540b-4cad-860b-5cfb8436a39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.039602 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c6e64cb6-540b-4cad-860b-5cfb8436a39f None None] MFA Rules not processed for user `b5e43c3f566f4647b98e8882dcf293e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:13.040826 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c6e64cb6-540b-4cad-860b-5cfb8436a39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:13.071415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:13.106759 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1261/2537] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:13.111286 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62746-2ab7-461f-80b1-16c5d14d6063 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:13.129487 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:13.175572 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62746-2ab7-461f-80b1-16c5d14d6063 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:13.180440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b4c8-267f-45f3-9f87-44fa8a91a16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:13.181273 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b4c8-267f-45f3-9f87-44fa8a91a16c admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.181758 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b4c8-267f-45f3-9f87-44fa8a91a16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.182374 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b4c8-267f-45f3-9f87-44fa8a91a16c admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.183021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b4c8-267f-45f3-9f87-44fa8a91a16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:13.184952 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b4c8-267f-45f3-9f87-44fa8a91a16c admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:13.257148 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1277/2538] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:13 2022] POST /identity/v3/projects => generated 357 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:13.261397 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e64cb6-540b-4cad-860b-5cfb8436a39f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:13.272255 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e64cb6-540b-4cad-860b-5cfb8436a39f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:13.328407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5880a-3780-4e5b-bde5-40afd610d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:13.329014 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5880a-3780-4e5b-bde5-40afd610d367 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.329290 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5880a-3780-4e5b-bde5-40afd610d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.329562 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5880a-3780-4e5b-bde5-40afd610d367 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.330202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5880a-3780-4e5b-bde5-40afd610d367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:13.332101 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5880a-3780-4e5b-bde5-40afd610d367 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:13.393046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1262/2539] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:13 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:13.397078 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b4c8-267f-45f3-9f87-44fa8a91a16c admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:13.407849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b4c8-267f-45f3-9f87-44fa8a91a16c admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:13.412494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b435fe-c6d3-4ced-b264-c6822ec17dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:13.413083 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b435fe-c6d3-4ced-b264-c6822ec17dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.413364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b435fe-c6d3-4ced-b264-c6822ec17dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.414281 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b435fe-c6d3-4ced-b264-c6822ec17dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.414922 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b435fe-c6d3-4ced-b264-c6822ec17dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:13.416536 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b435fe-c6d3-4ced-b264-c6822ec17dd7 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:13.427136 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1278/2540] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:13.431283 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5880a-3780-4e5b-bde5-40afd610d367 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:13.441417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5880a-3780-4e5b-bde5-40afd610d367 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:13.446182 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80201fb-26bf-4f25-a435-2777e09e7e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:13.446815 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80201fb-26bf-4f25-a435-2777e09e7e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.447091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80201fb-26bf-4f25-a435-2777e09e7e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.447361 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80201fb-26bf-4f25-a435-2777e09e7e13 admin admin] PATH_INFO: `/v3/projects/9f435e99c1484e2dbb71ffd30bcd52eb/users/c375bf9710a94c388188f549ad6267a1/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.462620 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80201fb-26bf-4f25-a435-2777e09e7e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f435e99c1484e2dbb71ffd30bcd52eb, user_id=c375bf9710a94c388188f549ad6267a1, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:13.465312 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80201fb-26bf-4f25-a435-2777e09e7e13 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:13.546843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1263/2541] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:13 2022] PUT /identity/v3/projects/9f435e99c1484e2dbb71ffd30bcd52eb/users/c375bf9710a94c388188f549ad6267a1/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:13.551038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b435fe-c6d3-4ced-b264-c6822ec17dd7 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:13.561446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b435fe-c6d3-4ced-b264-c6822ec17dd7 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:13.618661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c193e4-4860-4595-b59d-fa25efe4b232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:13.619352 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c193e4-4860-4595-b59d-fa25efe4b232 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.619631 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c193e4-4860-4595-b59d-fa25efe4b232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.619900 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c193e4-4860-4595-b59d-fa25efe4b232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.620576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c193e4-4860-4595-b59d-fa25efe4b232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:13.622162 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c193e4-4860-4595-b59d-fa25efe4b232 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:13.633007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1279/2542] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:13.637073 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80201fb-26bf-4f25-a435-2777e09e7e13 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:13.647435 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80201fb-26bf-4f25-a435-2777e09e7e13 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:13.652691 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:13.653327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.653606 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.653879 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] PATH_INFO: `/v3/projects/9f435e99c1484e2dbb71ffd30bcd52eb/users/c375bf9710a94c388188f549ad6267a1/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.655787 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f435e99c1484e2dbb71ffd30bcd52eb, user_id=c375bf9710a94c388188f549ad6267a1, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:13.658429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:13.668657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1264/2543] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:13 2022] PUT /identity/v3/projects/9f435e99c1484e2dbb71ffd30bcd52eb/users/c375bf9710a94c388188f549ad6267a1/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:13.678383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f8218-ffaa-4c07-904e-c04a43699b5e None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.678670 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f8218-ffaa-4c07-904e-c04a43699b5e None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.678949 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f8218-ffaa-4c07-904e-c04a43699b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.706586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-687f8218-ffaa-4c07-904e-c04a43699b5e None None] MFA Rules not processed for user `c375bf9710a94c388188f549ad6267a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:13.707780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-687f8218-ffaa-4c07-904e-c04a43699b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:13.774104 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1280/2544] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:13.785135 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:13.786822 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:13.842921 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b573585-5870-428f-a584-d86bf31178ae admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:13.846470 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9104d4-0063-4264-9691-d1e300cbcd0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:13.847046 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9104d4-0063-4264-9691-d1e300cbcd0c service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:13.847326 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9104d4-0063-4264-9691-d1e300cbcd0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:13.847637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9104d4-0063-4264-9691-d1e300cbcd0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:13.857310 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9104d4-0063-4264-9691-d1e300cbcd0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:13.858779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9104d4-0063-4264-9691-d1e300cbcd0c service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:13.868174 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1265/2545] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:13 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:14.063707 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f8218-ffaa-4c07-904e-c04a43699b5e None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:14.074458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f8218-ffaa-4c07-904e-c04a43699b5e None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:14.078105 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc516c8c-e400-407c-a242-02bbf679a63c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:14.078718 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc516c8c-e400-407c-a242-02bbf679a63c service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:14.078996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc516c8c-e400-407c-a242-02bbf679a63c service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:14.079266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc516c8c-e400-407c-a242-02bbf679a63c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:14.089570 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:14.137370 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc516c8c-e400-407c-a242-02bbf679a63c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:14.138790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc516c8c-e400-407c-a242-02bbf679a63c service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:14.147381 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1281/2546] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:23.863917 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:24.152857 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=23 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:32.281782 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9104d4-0063-4264-9691-d1e300cbcd0c service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:32.342303 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9104d4-0063-4264-9691-d1e300cbcd0c service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:32.346756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:32.347399 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:32.347675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:32.347943 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] PATH_INFO: `/v3/users/b5e43c3f566f4647b98e8882dcf293e7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:32.349088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e43c3f566f4647b98e8882dcf293e7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:32.350961 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:32.566110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e43c3f566f4647b98e8882dcf293e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:32.573779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e43c3f566f4647b98e8882dcf293e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:32.586128 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e43c3f566f4647b98e8882dcf293e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:32.649625 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e43c3f566f4647b98e8882dcf293e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:32.650237 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e43c3f566f4647b98e8882dcf293e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:32.656181 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1266/2547] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:32 2022] DELETE /identity/v3/users/b5e43c3f566f4647b98e8882dcf293e7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:32.661230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad337db-c18a-468c-9279-cc449b2dd1f7 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:32.661230 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad337db-c18a-468c-9279-cc449b2dd1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:32.661831 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad337db-c18a-468c-9279-cc449b2dd1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:32.688774 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-9ad337db-c18a-468c-9279-cc449b2dd1f7 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:32.689959 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9ad337db-c18a-468c-9279-cc449b2dd1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:32.750385 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1282/2548] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:32.760506 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:32.761888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:32.829773 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0651b-6741-489e-be40-a10b23fd3158 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:32.833204 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644408ba-8b54-4902-a5cb-4d6f2e7163d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:32.833788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644408ba-8b54-4902-a5cb-4d6f2e7163d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:32.834059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644408ba-8b54-4902-a5cb-4d6f2e7163d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:32.834324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644408ba-8b54-4902-a5cb-4d6f2e7163d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:32.843023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644408ba-8b54-4902-a5cb-4d6f2e7163d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:32.844621 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644408ba-8b54-4902-a5cb-4d6f2e7163d9 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:32.853819 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1267/2549] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:33.211274 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad337db-c18a-468c-9279-cc449b2dd1f7 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:33.212780 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-9ad337db-c18a-468c-9279-cc449b2dd1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:33.224759 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad337db-c18a-468c-9279-cc449b2dd1f7 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:33.229874 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:33.230499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:33.230777 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:33.231048 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] PATH_INFO: `/v3/projects/ca89ab47fdb64e67b070a60f9258ca31` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:33.232686 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca89ab47fdb64e67b070a60f9258ca31)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:33.234275 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:33.319119 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca89ab47fdb64e67b070a60f9258ca31', 'request_id': 'req-8843d8ec-28f2-4656-b09c-51ff0507c0da'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.398254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca89ab47fdb64e67b070a60f9258ca31', 'request_id': 'req-8843d8ec-28f2-4656-b09c-51ff0507c0da'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.405269 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca89ab47fdb64e67b070a60f9258ca31', 'request_id': 'req-8843d8ec-28f2-4656-b09c-51ff0507c0da'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.416406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] The token cache is being invalidate because project ca89ab47fdb64e67b070a60f9258ca31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:33.416694 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.417327 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.418789 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1283/2550] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:33 2022] DELETE /identity/v3/projects/ca89ab47fdb64e67b070a60f9258ca31 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:33.422744 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644408ba-8b54-4902-a5cb-4d6f2e7163d9 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:33.432494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644408ba-8b54-4902-a5cb-4d6f2e7163d9 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:33.440249 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:33.489567 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:33.490180 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:33.490488 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:33.490756 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] PATH_INFO: `/v3/users/c375bf9710a94c388188f549ad6267a1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:33.491860 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c375bf9710a94c388188f549ad6267a1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:33.493820 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:33.658517 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c375bf9710a94c388188f549ad6267a1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.666065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c375bf9710a94c388188f549ad6267a1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.678487 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c375bf9710a94c388188f549ad6267a1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.752779 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c375bf9710a94c388188f549ad6267a1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.753418 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb51f213-8b5c-44e4-b29e-ef7d843ba853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c375bf9710a94c388188f549ad6267a1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:33.759968 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1268/2551] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:33 2022] DELETE /identity/v3/users/c375bf9710a94c388188f549ad6267a1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:34.154130 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:34.155577 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:34.169124 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843d8ec-28f2-4656-b09c-51ff0507c0da admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:34.226460 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:34.227089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.227369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.227644 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] PATH_INFO: `/v3/projects/9f435e99c1484e2dbb71ffd30bcd52eb` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.228902 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f435e99c1484e2dbb71ffd30bcd52eb)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:34.230813 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:34.251514 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:34.312967 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f435e99c1484e2dbb71ffd30bcd52eb', 'request_id': 'req-27958f0e-587a-4013-b61a-d487b4159611'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:34.385998 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f435e99c1484e2dbb71ffd30bcd52eb', 'request_id': 'req-27958f0e-587a-4013-b61a-d487b4159611'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:34.392404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f435e99c1484e2dbb71ffd30bcd52eb', 'request_id': 'req-27958f0e-587a-4013-b61a-d487b4159611'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:34.402709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] The token cache is being invalidate because project 9f435e99c1484e2dbb71ffd30bcd52eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:34.402996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:34.403611 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-27958f0e-587a-4013-b61a-d487b4159611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:34.405094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1284/2552] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:34 2022] DELETE /identity/v3/projects/9f435e99c1484e2dbb71ffd30bcd52eb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:34.410088 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a1961-7b68-40de-93b9-222cc8f9dc23 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.410375 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a1961-7b68-40de-93b9-222cc8f9dc23 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.410651 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a1961-7b68-40de-93b9-222cc8f9dc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.458574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-eb4a1961-7b68-40de-93b9-222cc8f9dc23 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:34.459696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eb4a1961-7b68-40de-93b9-222cc8f9dc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:34.518354 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1269/2553] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:34.527129 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc302172-af4d-44aa-8187-ff0eb62070e4 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.532893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc302172-af4d-44aa-8187-ff0eb62070e4 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.532893 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc302172-af4d-44aa-8187-ff0eb62070e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.548010 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-dc302172-af4d-44aa-8187-ff0eb62070e4 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:34.549246 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-dc302172-af4d-44aa-8187-ff0eb62070e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:34.557662 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1285/2554] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:34.562215 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a1961-7b68-40de-93b9-222cc8f9dc23 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:34.572195 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a1961-7b68-40de-93b9-222cc8f9dc23 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:34.576797 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:34.577381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.577657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.577925 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.578560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:44:34.578863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:34.580518 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:34.581974 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:34.593685 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1270/2555] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:44:34.597948 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc302172-af4d-44aa-8187-ff0eb62070e4 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:34.607910 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc302172-af4d-44aa-8187-ff0eb62070e4 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:34.612630 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1b2b1-0bae-4227-bb74-196feca61e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:34.613219 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b2b1-0bae-4227-bb74-196feca61e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.613499 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b2b1-0bae-4227-bb74-196feca61e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.613769 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b2b1-0bae-4227-bb74-196feca61e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.614387 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1b2b1-0bae-4227-bb74-196feca61e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:34.616263 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1b2b1-0bae-4227-bb74-196feca61e28 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:34.720140 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1286/2556] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:34 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:34.724226 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:34.734668 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c92979-421e-48cb-9afb-af21599e2e27 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:34.790887 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61410f38-a82d-4b18-96bf-6b2a53104b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:34.791533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61410f38-a82d-4b18-96bf-6b2a53104b71 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.791803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61410f38-a82d-4b18-96bf-6b2a53104b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.792069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61410f38-a82d-4b18-96bf-6b2a53104b71 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.792721 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61410f38-a82d-4b18-96bf-6b2a53104b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:34.794630 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61410f38-a82d-4b18-96bf-6b2a53104b71 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:34.878179 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1271/2557] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:34 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:34.882081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1b2b1-0bae-4227-bb74-196feca61e28 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:34.892337 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1b2b1-0bae-4227-bb74-196feca61e28 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:34.897028 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159b5c-2bc0-474c-93e4-49024d0184a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:34.897613 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159b5c-2bc0-474c-93e4-49024d0184a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.897889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159b5c-2bc0-474c-93e4-49024d0184a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.898159 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159b5c-2bc0-474c-93e4-49024d0184a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.898793 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90159b5c-2bc0-474c-93e4-49024d0184a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:34.900916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90159b5c-2bc0-474c-93e4-49024d0184a6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:34.910976 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1287/2558] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:34.915024 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61410f38-a82d-4b18-96bf-6b2a53104b71 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:34.925637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61410f38-a82d-4b18-96bf-6b2a53104b71 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:34.930267 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce34807-908a-4f47-bae3-a3e9d231520b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:34.930855 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce34807-908a-4f47-bae3-a3e9d231520b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.931134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce34807-908a-4f47-bae3-a3e9d231520b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.931407 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce34807-908a-4f47-bae3-a3e9d231520b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.932049 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce34807-908a-4f47-bae3-a3e9d231520b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:34.933661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce34807-908a-4f47-bae3-a3e9d231520b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:34.943722 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1272/2559] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:34.947845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159b5c-2bc0-474c-93e4-49024d0184a6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:34.957829 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159b5c-2bc0-474c-93e4-49024d0184a6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:34.963100 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14fbb-f21a-4798-b5e5-3c4b981afadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:34.963756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14fbb-f21a-4798-b5e5-3c4b981afadd admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:34.964038 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14fbb-f21a-4798-b5e5-3c4b981afadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:34.964347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb14fbb-f21a-4798-b5e5-3c4b981afadd admin admin] PATH_INFO: `/v3/projects/b10812ee049e4673b5e62ab802716de5/users/1a94c82dd34c453f85aab6b6b3d4bae7/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:34.979472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb14fbb-f21a-4798-b5e5-3c4b981afadd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10812ee049e4673b5e62ab802716de5, user_id=1a94c82dd34c453f85aab6b6b3d4bae7, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:34.982094 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb14fbb-f21a-4798-b5e5-3c4b981afadd admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:35.038074 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1288/2560] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:34 2022] PUT /identity/v3/projects/b10812ee049e4673b5e62ab802716de5/users/1a94c82dd34c453f85aab6b6b3d4bae7/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:35.048803 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e7087-bcf5-4ff9-8f6d-f995effd08c7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.049091 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e7087-bcf5-4ff9-8f6d-f995effd08c7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.049377 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e7087-bcf5-4ff9-8f6d-f995effd08c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.077139 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-df1e7087-bcf5-4ff9-8f6d-f995effd08c7 None None] MFA Rules not processed for user `1a94c82dd34c453f85aab6b6b3d4bae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:35.078346 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-df1e7087-bcf5-4ff9-8f6d-f995effd08c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:35.143611 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1273/2561] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:35.148017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14fbb-f21a-4798-b5e5-3c4b981afadd admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:35.209145 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb14fbb-f21a-4798-b5e5-3c4b981afadd admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:35.214423 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1550560-4827-4501-acfd-9a20539aab8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:35.215014 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1550560-4827-4501-acfd-9a20539aab8f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.215292 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1550560-4827-4501-acfd-9a20539aab8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.215557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1550560-4827-4501-acfd-9a20539aab8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.216180 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1550560-4827-4501-acfd-9a20539aab8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:35.218091 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1550560-4827-4501-acfd-9a20539aab8f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:35.290708 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1289/2562] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:35 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:44:35.294772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e7087-bcf5-4ff9-8f6d-f995effd08c7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:35.305357 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e7087-bcf5-4ff9-8f6d-f995effd08c7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:35.365627 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25422999-95e7-4718-b1fe-ea29efe79cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:35.366229 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422999-95e7-4718-b1fe-ea29efe79cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.366508 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422999-95e7-4718-b1fe-ea29efe79cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.366781 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422999-95e7-4718-b1fe-ea29efe79cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.367408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25422999-95e7-4718-b1fe-ea29efe79cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:35.369352 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25422999-95e7-4718-b1fe-ea29efe79cd2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:35.378134 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:35.495518 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1274/2563] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:35 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:35.499383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1550560-4827-4501-acfd-9a20539aab8f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:35.510040 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1550560-4827-4501-acfd-9a20539aab8f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:35.514640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ac4b-4360-49d1-8f96-4d0f50901fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:35.517359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ac4b-4360-49d1-8f96-4d0f50901fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.517359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ac4b-4360-49d1-8f96-4d0f50901fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.517359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ac4b-4360-49d1-8f96-4d0f50901fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.517359 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152ac4b-4360-49d1-8f96-4d0f50901fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:35.517957 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152ac4b-4360-49d1-8f96-4d0f50901fc6 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:35.529090 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1290/2564] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:35.533097 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25422999-95e7-4718-b1fe-ea29efe79cd2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:35.543141 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25422999-95e7-4718-b1fe-ea29efe79cd2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:35.548207 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81c580-66d3-4844-aa40-ba398034d970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:35.548871 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81c580-66d3-4844-aa40-ba398034d970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.549148 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81c580-66d3-4844-aa40-ba398034d970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.549417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81c580-66d3-4844-aa40-ba398034d970 admin admin] PATH_INFO: `/v3/projects/747a63e527c24f908ff4212265b8e9bc/users/2942ce2118d549da92f3e4636f68375e/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.564107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81c580-66d3-4844-aa40-ba398034d970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747a63e527c24f908ff4212265b8e9bc, user_id=2942ce2118d549da92f3e4636f68375e, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:35.566745 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81c580-66d3-4844-aa40-ba398034d970 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:35.638539 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1275/2565] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:35 2022] PUT /identity/v3/projects/747a63e527c24f908ff4212265b8e9bc/users/2942ce2118d549da92f3e4636f68375e/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:44:35.642465 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ac4b-4360-49d1-8f96-4d0f50901fc6 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:35.652188 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ac4b-4360-49d1-8f96-4d0f50901fc6 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:35.664770 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:35.708187 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d61ddf-649c-4591-bd4d-2e240f556078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:35.708817 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d61ddf-649c-4591-bd4d-2e240f556078 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.709098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d61ddf-649c-4591-bd4d-2e240f556078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.709372 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d61ddf-649c-4591-bd4d-2e240f556078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.710000 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d61ddf-649c-4591-bd4d-2e240f556078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:35.711581 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d61ddf-649c-4591-bd4d-2e240f556078 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:35.721169 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1291/2566] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:44:35.725408 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81c580-66d3-4844-aa40-ba398034d970 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:35.735652 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81c580-66d3-4844-aa40-ba398034d970 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:35.741793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:35.741793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.741793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.741793 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] PATH_INFO: `/v3/projects/747a63e527c24f908ff4212265b8e9bc/users/2942ce2118d549da92f3e4636f68375e/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.744089 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747a63e527c24f908ff4212265b8e9bc, user_id=2942ce2118d549da92f3e4636f68375e, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:35.746767 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:35.756761 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1276/2567] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:35 2022] PUT /identity/v3/projects/747a63e527c24f908ff4212265b8e9bc/users/2942ce2118d549da92f3e4636f68375e/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:35.768306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc48e3-8891-49ed-9c90-2ef6fbac3e1d None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.770515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc48e3-8891-49ed-9c90-2ef6fbac3e1d None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.770515 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc48e3-8891-49ed-9c90-2ef6fbac3e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.799127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-77fc48e3-8891-49ed-9c90-2ef6fbac3e1d None None] MFA Rules not processed for user `2942ce2118d549da92f3e4636f68375e`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:35.799127 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-77fc48e3-8891-49ed-9c90-2ef6fbac3e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:35.863306 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1292/2568] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:35.874416 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:35.876065 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:35.932851 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2d6c6-8af8-4018-bf68-d815f6f5a7c6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:35.936669 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c40b68-f678-4b0c-96f9-a6ffb1e9ab6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:35.937258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c40b68-f678-4b0c-96f9-a6ffb1e9ab6a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:35.937534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c40b68-f678-4b0c-96f9-a6ffb1e9ab6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:35.937802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c40b68-f678-4b0c-96f9-a6ffb1e9ab6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:35.947494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c40b68-f678-4b0c-96f9-a6ffb1e9ab6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:35.948930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c40b68-f678-4b0c-96f9-a6ffb1e9ab6a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:35.958148 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1277/2569] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:38.694102 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc48e3-8891-49ed-9c90-2ef6fbac3e1d None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:38.756440 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc48e3-8891-49ed-9c90-2ef6fbac3e1d None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:38.760838 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:38.761446 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:38.761721 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:38.762016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] PATH_INFO: `/v3/users/1a94c82dd34c453f85aab6b6b3d4bae7` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:38.763125 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a94c82dd34c453f85aab6b6b3d4bae7)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:38.765109 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:38.943953 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:38.949271 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a94c82dd34c453f85aab6b6b3d4bae7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:38.950011 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a94c82dd34c453f85aab6b6b3d4bae7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:38.961996 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a94c82dd34c453f85aab6b6b3d4bae7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:38.969437 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a94c82dd34c453f85aab6b6b3d4bae7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:38.976483 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a94c82dd34c453f85aab6b6b3d4bae7'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:39.048735 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1293/2570] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:38 2022] DELETE /identity/v3/users/1a94c82dd34c453f85aab6b6b3d4bae7 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:39.053127 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0fe87-0cbc-453b-ab69-d192b844d890 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:39.053656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0fe87-0cbc-453b-ab69-d192b844d890 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:39.053656 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0fe87-0cbc-453b-ab69-d192b844d890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:39.074447 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-20a0fe87-0cbc-453b-ab69-d192b844d890 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:39.075637 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-20a0fe87-0cbc-453b-ab69-d192b844d890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:39.133635 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1278/2571] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:39.143805 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:39.145260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:39.200467 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60758fcb-c2c7-4535-88fc-e7139f60ce99 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:39.204004 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db345a-1b37-42b0-a942-7b0e4c6d8d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:39.204615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db345a-1b37-42b0-a942-7b0e4c6d8d7c service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:39.204894 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db345a-1b37-42b0-a942-7b0e4c6d8d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:39.205163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db345a-1b37-42b0-a942-7b0e4c6d8d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:39.214393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db345a-1b37-42b0-a942-7b0e4c6d8d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:39.215790 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db345a-1b37-42b0-a942-7b0e4c6d8d7c service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:39.224815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1294/2572] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:39.612191 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0fe87-0cbc-453b-ab69-d192b844d890 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:39.614223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-20a0fe87-0cbc-453b-ab69-d192b844d890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:39.627903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0fe87-0cbc-453b-ab69-d192b844d890 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:39.632458 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:39.633080 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:39.633499 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:39.633772 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] PATH_INFO: `/v3/projects/b10812ee049e4673b5e62ab802716de5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:39.636069 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10812ee049e4673b5e62ab802716de5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:39.638293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:39.718706 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10812ee049e4673b5e62ab802716de5', 'request_id': 'req-3abfb277-7496-4b53-9f6d-9f51674b8660'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:39.720564 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:39.724496 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10812ee049e4673b5e62ab802716de5', 'request_id': 'req-3abfb277-7496-4b53-9f6d-9f51674b8660'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:39.731305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10812ee049e4673b5e62ab802716de5', 'request_id': 'req-3abfb277-7496-4b53-9f6d-9f51674b8660'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:39.785536 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] The token cache is being invalidate because project b10812ee049e4673b5e62ab802716de5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:39.785818 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:39.786417 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:39.787903 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1279/2573] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:39 2022] DELETE /identity/v3/projects/b10812ee049e4673b5e62ab802716de5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:39.792486 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db345a-1b37-42b0-a942-7b0e4c6d8d7c service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:39.802383 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db345a-1b37-42b0-a942-7b0e4c6d8d7c service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:39.859089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:39.859722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:39.860008 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:39.860307 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] PATH_INFO: `/v3/users/2942ce2118d549da92f3e4636f68375e` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:39.861456 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2942ce2118d549da92f3e4636f68375e)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:39.863393 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:40.011978 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2942ce2118d549da92f3e4636f68375e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.012661 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2942ce2118d549da92f3e4636f68375e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.023384 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:40.024656 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2942ce2118d549da92f3e4636f68375e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.031408 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2942ce2118d549da92f3e4636f68375e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.032036 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-7d4366c2-c31d-478b-83b6-db8943441fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2942ce2118d549da92f3e4636f68375e'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.106889 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1295/2574] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:44:39 2022] DELETE /identity/v3/users/2942ce2118d549da92f3e4636f68375e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:40.501557 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:40.503254 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:40.517575 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfb277-7496-4b53-9f6d-9f51674b8660 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:40.576479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:40.577105 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:40.577388 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:40.577666 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] PATH_INFO: `/v3/projects/747a63e527c24f908ff4212265b8e9bc` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:40.578891 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747a63e527c24f908ff4212265b8e9bc)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:40.580858 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:40.706516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747a63e527c24f908ff4212265b8e9bc', 'request_id': 'req-241a7805-9712-496e-bd78-f8289e6aed68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.712728 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747a63e527c24f908ff4212265b8e9bc', 'request_id': 'req-241a7805-9712-496e-bd78-f8289e6aed68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.719057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747a63e527c24f908ff4212265b8e9bc', 'request_id': 'req-241a7805-9712-496e-bd78-f8289e6aed68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.791699 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] The token cache is being invalidate because project 747a63e527c24f908ff4212265b8e9bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:44:40.791907 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.792527 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-241a7805-9712-496e-bd78-f8289e6aed68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:44:40.794009 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1280/2575] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:44:40 2022] DELETE /identity/v3/projects/747a63e527c24f908ff4212265b8e9bc => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:40.798791 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d91e-1cee-4cb5-aa13-7ba8b2eddf46 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:40.799266 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d91e-1cee-4cb5-aa13-7ba8b2eddf46 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:40.799391 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6d91e-1cee-4cb5-aa13-7ba8b2eddf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:40.820117 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d0f6d91e-1cee-4cb5-aa13-7ba8b2eddf46 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:40.821320 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d0f6d91e-1cee-4cb5-aa13-7ba8b2eddf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:40.878674 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1296/2576] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:40.887040 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea9d83-a3f9-421e-a053-e658abfab291 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:40.887318 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea9d83-a3f9-421e-a053-e658abfab291 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:40.887586 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea9d83-a3f9-421e-a053-e658abfab291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:40.906885 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-55ea9d83-a3f9-421e-a053-e658abfab291 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:40.908092 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-55ea9d83-a3f9-421e-a053-e658abfab291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:40.915733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1281/2577] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:40.919756 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d91e-1cee-4cb5-aa13-7ba8b2eddf46 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:40.929043 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6d91e-1cee-4cb5-aa13-7ba8b2eddf46 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:40.933979 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:40.934554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:40.934826 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:40.935092 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:40.935722 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:44:40.936020 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:40.937615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:40.949639 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1297/2578] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:44:40.953648 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea9d83-a3f9-421e-a053-e658abfab291 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:40.962534 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea9d83-a3f9-421e-a053-e658abfab291 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:40.967035 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d85d8-335b-4380-8e92-6a9e1e2e8a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:40.967608 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d85d8-335b-4380-8e92-6a9e1e2e8a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:40.967880 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d85d8-335b-4380-8e92-6a9e1e2e8a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:40.968146 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d85d8-335b-4380-8e92-6a9e1e2e8a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:40.968802 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d85d8-335b-4380-8e92-6a9e1e2e8a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:40.970654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d85d8-335b-4380-8e92-6a9e1e2e8a78 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.079829 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1282/2579] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:40 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:41.083870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.093821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73f199-cbb9-4331-920f-92e18ebb2447 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:41.149768 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8331529-6f7d-461e-aed7-e8947d91b137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:41.150362 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8331529-6f7d-461e-aed7-e8947d91b137 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.150640 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8331529-6f7d-461e-aed7-e8947d91b137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.150944 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8331529-6f7d-461e-aed7-e8947d91b137 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.151586 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8331529-6f7d-461e-aed7-e8947d91b137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:41.153509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8331529-6f7d-461e-aed7-e8947d91b137 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.226209 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1298/2580] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:41 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:41.231027 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d85d8-335b-4380-8e92-6a9e1e2e8a78 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.241264 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d85d8-335b-4380-8e92-6a9e1e2e8a78 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:41.246504 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc670e-7e7a-41f8-8233-b23400e3bf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:41.247082 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc670e-7e7a-41f8-8233-b23400e3bf27 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.247364 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc670e-7e7a-41f8-8233-b23400e3bf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.247634 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc670e-7e7a-41f8-8233-b23400e3bf27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.248257 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc670e-7e7a-41f8-8233-b23400e3bf27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:41.249843 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc670e-7e7a-41f8-8233-b23400e3bf27 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.259498 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1283/2581] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:41.263260 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8331529-6f7d-461e-aed7-e8947d91b137 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.273143 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8331529-6f7d-461e-aed7-e8947d91b137 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:41.277847 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1eb2d-6d56-406a-a0e6-e5d81fa969e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:41.278432 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1eb2d-6d56-406a-a0e6-e5d81fa969e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.278713 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1eb2d-6d56-406a-a0e6-e5d81fa969e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.278987 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1eb2d-6d56-406a-a0e6-e5d81fa969e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.279615 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1eb2d-6d56-406a-a0e6-e5d81fa969e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:41.281238 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1eb2d-6d56-406a-a0e6-e5d81fa969e3 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.292345 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1299/2582] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:41.296689 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc670e-7e7a-41f8-8233-b23400e3bf27 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.307516 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc670e-7e7a-41f8-8233-b23400e3bf27 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:41.312114 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09e9de-2a76-4614-879d-9223124813ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:41.319657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09e9de-2a76-4614-879d-9223124813ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.319657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09e9de-2a76-4614-879d-9223124813ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.319657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09e9de-2a76-4614-879d-9223124813ed admin admin] PATH_INFO: `/v3/projects/2c1d1a24af8b4e0685d8efa8c8f7630f/users/3722319df7644b388e177ad1d92177ad/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.327957 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09e9de-2a76-4614-879d-9223124813ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1d1a24af8b4e0685d8efa8c8f7630f, user_id=3722319df7644b388e177ad1d92177ad, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:41.330696 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09e9de-2a76-4614-879d-9223124813ed admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.397996 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1284/2583] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:41 2022] PUT /identity/v3/projects/2c1d1a24af8b4e0685d8efa8c8f7630f/users/3722319df7644b388e177ad1d92177ad/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:44:41.408349 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329923-30c4-4e4f-b7f2-05f445bc64e6 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.408635 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329923-30c4-4e4f-b7f2-05f445bc64e6 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.408907 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329923-30c4-4e4f-b7f2-05f445bc64e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.436983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-05329923-30c4-4e4f-b7f2-05f445bc64e6 None None] MFA Rules not processed for user `3722319df7644b388e177ad1d92177ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:41.438157 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-05329923-30c4-4e4f-b7f2-05f445bc64e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:41.469696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:41.499734 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1300/2584] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:41.504178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09e9de-2a76-4614-879d-9223124813ed admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.513169 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:41.565309 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09e9de-2a76-4614-879d-9223124813ed admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:41.570223 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d067236-ba6c-404a-b937-366f44afa358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:41.570821 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d067236-ba6c-404a-b937-366f44afa358 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.571107 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d067236-ba6c-404a-b937-366f44afa358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.571381 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d067236-ba6c-404a-b937-366f44afa358 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.572006 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d067236-ba6c-404a-b937-366f44afa358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:41.573939 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d067236-ba6c-404a-b937-366f44afa358 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.662246 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1285/2585] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:44:41 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:41.666971 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05329923-30c4-4e4f-b7f2-05f445bc64e6 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.677773 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05329923-30c4-4e4f-b7f2-05f445bc64e6 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:41.733052 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec3cd-2bc9-426d-bebe-cf3ccdafc615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:41.733642 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec3cd-2bc9-426d-bebe-cf3ccdafc615 admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.733936 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec3cd-2bc9-426d-bebe-cf3ccdafc615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.734183 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec3cd-2bc9-426d-bebe-cf3ccdafc615 admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.734822 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aec3cd-2bc9-426d-bebe-cf3ccdafc615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:41.736748 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aec3cd-2bc9-426d-bebe-cf3ccdafc615 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.821133 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1301/2586] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:44:41 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:44:41.825102 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d067236-ba6c-404a-b937-366f44afa358 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.835489 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d067236-ba6c-404a-b937-366f44afa358 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:41.840809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651fd8d-ae29-44ea-b825-1c00267d085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:41.841380 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651fd8d-ae29-44ea-b825-1c00267d085d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.841657 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651fd8d-ae29-44ea-b825-1c00267d085d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.841927 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651fd8d-ae29-44ea-b825-1c00267d085d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.842582 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651fd8d-ae29-44ea-b825-1c00267d085d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:41.844149 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651fd8d-ae29-44ea-b825-1c00267d085d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.855276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1286/2587] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:44:41.859300 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec3cd-2bc9-426d-bebe-cf3ccdafc615 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.869198 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec3cd-2bc9-426d-bebe-cf3ccdafc615 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:41.873719 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcb84f-d4af-497c-b69d-9b5cf62450df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:41.874342 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcb84f-d4af-497c-b69d-9b5cf62450df admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:41.874617 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcb84f-d4af-497c-b69d-9b5cf62450df admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:41.874885 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbcb84f-d4af-497c-b69d-9b5cf62450df admin admin] PATH_INFO: `/v3/projects/6c257216de12439eb6b007077dcf02b7/users/52ffca1752cc497fb83f48a5cce33752/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:41.889012 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcb84f-d4af-497c-b69d-9b5cf62450df admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c257216de12439eb6b007077dcf02b7, user_id=52ffca1752cc497fb83f48a5cce33752, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:41.891637 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbcb84f-d4af-497c-b69d-9b5cf62450df admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:41.952452 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1302/2588] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:41 2022] PUT /identity/v3/projects/6c257216de12439eb6b007077dcf02b7/users/52ffca1752cc497fb83f48a5cce33752/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:41.956235 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651fd8d-ae29-44ea-b825-1c00267d085d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:41.966914 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651fd8d-ae29-44ea-b825-1c00267d085d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:42.021846 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7016bd8-31af-42ed-99af-de6ffc02fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:42.022442 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7016bd8-31af-42ed-99af-de6ffc02fb40 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:42.022720 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7016bd8-31af-42ed-99af-de6ffc02fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:42.022993 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7016bd8-31af-42ed-99af-de6ffc02fb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:42.023616 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7016bd8-31af-42ed-99af-de6ffc02fb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:42.025896 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7016bd8-31af-42ed-99af-de6ffc02fb40 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:42.035335 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1287/2589] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:44:42.039351 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcb84f-d4af-497c-b69d-9b5cf62450df admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:42.049472 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbcb84f-d4af-497c-b69d-9b5cf62450df admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:42.053938 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:42.054567 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:42.054839 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:42.055106 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] PATH_INFO: `/v3/projects/6c257216de12439eb6b007077dcf02b7/users/52ffca1752cc497fb83f48a5cce33752/roles/15df2a7bb32b484894972c74c547f5a4` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:42.057029 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c257216de12439eb6b007077dcf02b7, user_id=52ffca1752cc497fb83f48a5cce33752, role_id=15df2a7bb32b484894972c74c547f5a4)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:42.059645 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:42.069871 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1303/2590] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:44:42 2022] PUT /identity/v3/projects/6c257216de12439eb6b007077dcf02b7/users/52ffca1752cc497fb83f48a5cce33752/roles/15df2a7bb32b484894972c74c547f5a4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:44:42.081258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1eba-5c12-4285-a63f-47d75ce0c589 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:42.081542 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1eba-5c12-4285-a63f-47d75ce0c589 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:42.081817 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1eba-5c12-4285-a63f-47d75ce0c589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:42.109643 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7e2f1eba-5c12-4285-a63f-47d75ce0c589 None None] MFA Rules not processed for user `52ffca1752cc497fb83f48a5cce33752`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:44:42.110838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7e2f1eba-5c12-4285-a63f-47d75ce0c589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:42.178301 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1288/2591] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:44:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:44:42.189174 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:42.190833 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:42.267804 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad73595-1afa-4427-9979-52116eb27fa1 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:42.271307 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2984651-57cd-44ad-b6e4-758cf2617a62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:42.271891 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2984651-57cd-44ad-b6e4-758cf2617a62 service cinder] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:42.272163 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2984651-57cd-44ad-b6e4-758cf2617a62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:42.272457 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2984651-57cd-44ad-b6e4-758cf2617a62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:42.292918 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:42.328494 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2984651-57cd-44ad-b6e4-758cf2617a62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:42.330325 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2984651-57cd-44ad-b6e4-758cf2617a62 service cinder] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:42.339824 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1304/2592] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:48.593178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f1eba-5c12-4285-a63f-47d75ce0c589 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:44:48.594584 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7e2f1eba-5c12-4285-a63f-47d75ce0c589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:44:48.601401 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:48.606850 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f1eba-5c12-4285-a63f-47d75ce0c589 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:44:48.610384 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ccfab-460c-47ce-b488-b6f7e0fd2311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:44:48.616413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ccfab-460c-47ce-b488-b6f7e0fd2311 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:44:48.616413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ccfab-460c-47ce-b488-b6f7e0fd2311 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:44:48.616413 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ccfab-460c-47ce-b488-b6f7e0fd2311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:44:48.620103 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ccfab-460c-47ce-b488-b6f7e0fd2311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:44:48.622042 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ccfab-460c-47ce-b488-b6f7e0fd2311 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:44:48.631469 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1289/2593] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:44:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:44:52.357276 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=17 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:44:58.648899 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:19.514430 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2984651-57cd-44ad-b6e4-758cf2617a62 service cinder] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:19.575953 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2984651-57cd-44ad-b6e4-758cf2617a62 service cinder] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:19.581093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:19.581709 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:19.581984 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:19.582254 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] PATH_INFO: `/v3/users/3722319df7644b388e177ad1d92177ad` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:19.583334 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3722319df7644b388e177ad1d92177ad)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:19.585297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:19.793347 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3722319df7644b388e177ad1d92177ad'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:19.793917 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3722319df7644b388e177ad1d92177ad'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:19.805851 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3722319df7644b388e177ad1d92177ad'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:19.813189 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3722319df7644b388e177ad1d92177ad'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:19.813816 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3722319df7644b388e177ad1d92177ad'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:19.877378 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1305/2594] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:45:19 2022] DELETE /identity/v3/users/3722319df7644b388e177ad1d92177ad => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:45:19.881788 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fd25f-c089-4e71-8b22-93affd25d354 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:19.883178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fd25f-c089-4e71-8b22-93affd25d354 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:19.883178 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fd25f-c089-4e71-8b22-93affd25d354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:19.901841 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-160fd25f-c089-4e71-8b22-93affd25d354 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:19.903034 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-160fd25f-c089-4e71-8b22-93affd25d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:19.960739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1290/2595] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:19.970289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:19.971672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:20.027696 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673062d-1499-4672-91eb-d1c339feb845 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:20.031226 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a7580-4001-4085-b350-52aba6c961f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:20.031815 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a7580-4001-4085-b350-52aba6c961f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:20.032093 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a7580-4001-4085-b350-52aba6c961f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:20.032395 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a7580-4001-4085-b350-52aba6c961f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:20.041139 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a7580-4001-4085-b350-52aba6c961f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:20.042552 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a7580-4001-4085-b350-52aba6c961f0 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:20.052369 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1306/2596] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:45:20.473673 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fd25f-c089-4e71-8b22-93affd25d354 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:20.477324 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-160fd25f-c089-4e71-8b22-93affd25d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:20.491712 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fd25f-c089-4e71-8b22-93affd25d354 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:20.496605 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:20.500280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:20.500280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:20.500280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] PATH_INFO: `/v3/projects/2c1d1a24af8b4e0685d8efa8c8f7630f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:20.500280 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1d1a24af8b4e0685d8efa8c8f7630f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:20.501577 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:20.615415 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1d1a24af8b4e0685d8efa8c8f7630f', 'request_id': 'req-e8a97af7-2cc0-455f-a13c-4136fed3c43a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.621137 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1d1a24af8b4e0685d8efa8c8f7630f', 'request_id': 'req-e8a97af7-2cc0-455f-a13c-4136fed3c43a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.627556 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1d1a24af8b4e0685d8efa8c8f7630f', 'request_id': 'req-e8a97af7-2cc0-455f-a13c-4136fed3c43a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.693574 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] The token cache is being invalidate because project 2c1d1a24af8b4e0685d8efa8c8f7630f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:45:20.693902 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.694505 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.695988 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1291/2597] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:45:20 2022] DELETE /identity/v3/projects/2c1d1a24af8b4e0685d8efa8c8f7630f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:45:20.699943 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a7580-4001-4085-b350-52aba6c961f0 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:20.709458 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a7580-4001-4085-b350-52aba6c961f0 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:20.727573 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:20.765082 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:20.765699 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:20.766476 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:20.766741 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] PATH_INFO: `/v3/users/52ffca1752cc497fb83f48a5cce33752` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:20.767897 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ffca1752cc497fb83f48a5cce33752)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:20.769849 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:20.954193 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ffca1752cc497fb83f48a5cce33752'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.954845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ffca1752cc497fb83f48a5cce33752'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.966905 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ffca1752cc497fb83f48a5cce33752'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.974361 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ffca1752cc497fb83f48a5cce33752'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:20.974991 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-9fc2eb18-038e-4b36-a454-3b35e74a6b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ffca1752cc497fb83f48a5cce33752'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:21.042364 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1307/2598] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:45:20 2022] DELETE /identity/v3/users/52ffca1752cc497fb83f48a5cce33752 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:45:21.446033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:21.448400 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:21.462763 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97af7-2cc0-455f-a13c-4136fed3c43a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:21.520992 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:21.521650 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:21.521930 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:21.522205 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] PATH_INFO: `/v3/projects/6c257216de12439eb6b007077dcf02b7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:21.523306 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c257216de12439eb6b007077dcf02b7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:21.525281 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:21.644328 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:21.660293 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c257216de12439eb6b007077dcf02b7', 'request_id': 'req-8b43958c-8e84-4148-986e-da8ddf8732c0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:21.665739 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c257216de12439eb6b007077dcf02b7', 'request_id': 'req-8b43958c-8e84-4148-986e-da8ddf8732c0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:21.672494 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c257216de12439eb6b007077dcf02b7', 'request_id': 'req-8b43958c-8e84-4148-986e-da8ddf8732c0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:21.721661 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] The token cache is being invalidate because project 6c257216de12439eb6b007077dcf02b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:45:21.721866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:21.722478 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8b43958c-8e84-4148-986e-da8ddf8732c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:21.723936 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1292/2599] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:45:21 2022] DELETE /identity/v3/projects/6c257216de12439eb6b007077dcf02b7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:45:21.732464 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db547c-e146-4ec1-83fb-c104c3f2c47f None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:21.733044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db547c-e146-4ec1-83fb-c104c3f2c47f None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:21.733044 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db547c-e146-4ec1-83fb-c104c3f2c47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:21.753801 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-31db547c-e146-4ec1-83fb-c104c3f2c47f None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:21.755022 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-31db547c-e146-4ec1-83fb-c104c3f2c47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:21.810830 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1308/2600] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:21.819639 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e82d7b-d0ea-4491-8c1a-2e54913ea384 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:21.819915 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e82d7b-d0ea-4491-8c1a-2e54913ea384 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:21.820184 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e82d7b-d0ea-4491-8c1a-2e54913ea384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:21.839313 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a2e82d7b-d0ea-4491-8c1a-2e54913ea384 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:21.840507 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a2e82d7b-d0ea-4491-8c1a-2e54913ea384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:21.848132 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1293/2601] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:21.852165 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db547c-e146-4ec1-83fb-c104c3f2c47f None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:21.862076 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db547c-e146-4ec1-83fb-c104c3f2c47f None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:21.866603 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:21.867186 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:21.867462 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:21.867733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] PATH_INFO: `/v3/domains` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:21.868398 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:45:21.868725 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:21.870284 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:21.882406 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1309/2602] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Oct 24 13:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:45:21.886355 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e82d7b-d0ea-4491-8c1a-2e54913ea384 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:21.896928 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e82d7b-d0ea-4491-8c1a-2e54913ea384 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:21.901548 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d35e53-bb50-4015-9e9c-ffecbc9393d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:21.902130 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d35e53-bb50-4015-9e9c-ffecbc9393d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:21.902404 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d35e53-bb50-4015-9e9c-ffecbc9393d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:21.902672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d35e53-bb50-4015-9e9c-ffecbc9393d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:21.903292 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d35e53-bb50-4015-9e9c-ffecbc9393d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:21.905201 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d35e53-bb50-4015-9e9c-ffecbc9393d4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:22.009231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1294/2603] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Oct 24 13:45:21 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:45:22.013265 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:22.023099 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17380569-dd21-4717-a297-ee8c14019364 admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:22.049846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:22.080060 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dffdad-240e-47da-819c-ef6747005e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:22.080693 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dffdad-240e-47da-819c-ef6747005e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:22.080970 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dffdad-240e-47da-819c-ef6747005e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:22.081240 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dffdad-240e-47da-819c-ef6747005e2f admin admin] PATH_INFO: `/v3/users` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:22.081870 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dffdad-240e-47da-819c-ef6747005e2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:22.083919 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dffdad-240e-47da-819c-ef6747005e2f admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:22.196846 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1310/2604] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Oct 24 13:45:22 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:45:22.201055 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d35e53-bb50-4015-9e9c-ffecbc9393d4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:22.211202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d35e53-bb50-4015-9e9c-ffecbc9393d4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:22.215662 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217863-ad8c-4406-ac9b-2a0ed440779d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:22.216239 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217863-ad8c-4406-ac9b-2a0ed440779d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:22.216544 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217863-ad8c-4406-ac9b-2a0ed440779d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:22.216809 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217863-ad8c-4406-ac9b-2a0ed440779d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:22.217433 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16217863-ad8c-4406-ac9b-2a0ed440779d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:22.219538 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16217863-ad8c-4406-ac9b-2a0ed440779d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:22.229281 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1295/2605] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:45:22.232988 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dffdad-240e-47da-819c-ef6747005e2f admin admin] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:22.243521 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dffdad-240e-47da-819c-ef6747005e2f admin admin] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:22.248141 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edecd3-178c-4aed-acb0-5ebaab67bf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:22.248739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edecd3-178c-4aed-acb0-5ebaab67bf45 admin admin] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:22.249016 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edecd3-178c-4aed-acb0-5ebaab67bf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:22.249289 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edecd3-178c-4aed-acb0-5ebaab67bf45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:22.249915 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edecd3-178c-4aed-acb0-5ebaab67bf45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:22.251529 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edecd3-178c-4aed-acb0-5ebaab67bf45 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:22.261350 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1311/2606] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Oct 24 13:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:45:22.265158 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217863-ad8c-4406-ac9b-2a0ed440779d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:22.275393 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217863-ad8c-4406-ac9b-2a0ed440779d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:22.279854 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:22.280561 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:22.280837 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:22.281106 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] PATH_INFO: `/v3/projects/0cf2f5ececed49adb39332338164ec3f/users/a0b551495dca4982bef194a97c707fb5/roles/eb33c965778e405b95ebf794f31e329a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:22.296084 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf2f5ececed49adb39332338164ec3f, user_id=a0b551495dca4982bef194a97c707fb5, role_id=eb33c965778e405b95ebf794f31e329a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:22.298714 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:22.379234 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1296/2607] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Oct 24 13:45:22 2022] PUT /identity/v3/projects/0cf2f5ececed49adb39332338164ec3f/users/a0b551495dca4982bef194a97c707fb5/roles/eb33c965778e405b95ebf794f31e329a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:45:22.391428 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b6742-9dc4-499a-adb7-903d4b721a2a None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:22.391716 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b6742-9dc4-499a-adb7-903d4b721a2a None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:22.391992 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b6742-9dc4-499a-adb7-903d4b721a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:22.419890 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-5c4b6742-9dc4-499a-adb7-903d4b721a2a None None] MFA Rules not processed for user `a0b551495dca4982bef194a97c707fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:22.421108 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-5c4b6742-9dc4-499a-adb7-903d4b721a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:22.483576 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1312/2608] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:45:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:22.494133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:22.495897 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:22.553258 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa36d4-e800-4e56-898a-bc893cd3965c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:22.556978 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32fa3c94cad04287aedea87400a15591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:22.557576 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:22.557852 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:22.558119 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:22.566279 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:22.570923 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:22.576550 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1297/2609] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:45:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:45:24.374751 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2de3d-7f82-4fb8-9799-de516ba31365 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:24.375330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2de3d-7f82-4fb8-9799-de516ba31365 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:24.375330 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2de3d-7f82-4fb8-9799-de516ba31365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:24.394377 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-57b2de3d-7f82-4fb8-9799-de516ba31365 None None] MFA Rules not processed for user `a0b551495dca4982bef194a97c707fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:24.395554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-57b2de3d-7f82-4fb8-9799-de516ba31365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:24.402244 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1313/2610] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:24.440958 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:24.442672 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:24.489340 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:24.519700 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdbb20-c56e-445f-98aa-53203292431e service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:24.523334 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b2334-a60b-441e-ac28-cc1ad0169d53 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2800cf7f744a8973d5c5ca3907af1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:24.523926 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2334-a60b-441e-ac28-cc1ad0169d53 service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:24.524202 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2334-a60b-441e-ac28-cc1ad0169d53 service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:24.524500 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b2334-a60b-441e-ac28-cc1ad0169d53 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:24.526167 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-127b2334-a60b-441e-ac28-cc1ad0169d53 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:24.737372 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b2334-a60b-441e-ac28-cc1ad0169d53 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:24.738838 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b2334-a60b-441e-ac28-cc1ad0169d53 service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:24.758912 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1298/2611] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:45:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:45:25.531296 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2de3d-7f82-4fb8-9799-de516ba31365 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:25.532786 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-57b2de3d-7f82-4fb8-9799-de516ba31365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:25.589606 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2de3d-7f82-4fb8-9799-de516ba31365 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:25.593146 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:25.593733 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:25.594087 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:25.594357 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:25.596208 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:25.609415 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:25.610821 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:25.622017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1314/2612] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:45:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:45:34.791597 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=55 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:35.617299 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:45.755775 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3ee5b-3d5a-43dc-b207-eb047e0a24a4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:45.756323 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3ee5b-3d5a-43dc-b207-eb047e0a24a4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:45.756453 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3ee5b-3d5a-43dc-b207-eb047e0a24a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:45.786078 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e2b3ee5b-3d5a-43dc-b207-eb047e0a24a4 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:45.787319 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e2b3ee5b-3d5a-43dc-b207-eb047e0a24a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:45.841231 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1299/2613] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:45:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:45.993916 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:45.995297 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:46.075081 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9738fc-e5ae-4233-98b4-785fe357dbf4 service neutron] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:46.079929 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794e369-36b0-43a9-bdfa-19e0c564a767 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:46.080554 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794e369-36b0-43a9-bdfa-19e0c564a767 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:46.080927 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794e369-36b0-43a9-bdfa-19e0c564a767 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:46.081111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794e369-36b0-43a9-bdfa-19e0c564a767 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:46.090492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794e369-36b0-43a9-bdfa-19e0c564a767 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:46.092098 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794e369-36b0-43a9-bdfa-19e0c564a767 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:46.100466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1315/2614] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:45:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:45:46.891955 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c86d-b982-4bef-85ae-5400c5313cf3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:46.892533 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c86d-b982-4bef-85ae-5400c5313cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:46.892679 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c86d-b982-4bef-85ae-5400c5313cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:46.912836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a495c86d-b982-4bef-85ae-5400c5313cf3 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:46.914059 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a495c86d-b982-4bef-85ae-5400c5313cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:46.921110 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1300/2615] 10.222.0.48 () {58 vars in 967 bytes} [Mon Oct 24 13:45:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:46.933466 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794e369-36b0-43a9-bdfa-19e0c564a767 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:46.945034 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794e369-36b0-43a9-bdfa-19e0c564a767 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:46.950160 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c4e3e-cde9-4bc6-9062-4847b1774b87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:46.950739 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c4e3e-cde9-4bc6-9062-4847b1774b87 service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:46.951021 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c4e3e-cde9-4bc6-9062-4847b1774b87 service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:46.951293 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c4e3e-cde9-4bc6-9062-4847b1774b87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:46.959752 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c4e3e-cde9-4bc6-9062-4847b1774b87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:46.961361 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c4e3e-cde9-4bc6-9062-4847b1774b87 service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:46.969190 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1316/2616] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:45:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:45:47.686432 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be78c-a41b-49b0-9d40-fae1ebb4a698 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:47.686749 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be78c-a41b-49b0-9d40-fae1ebb4a698 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:47.687023 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be78c-a41b-49b0-9d40-fae1ebb4a698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:47.706402 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-022be78c-a41b-49b0-9d40-fae1ebb4a698 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:47.707580 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-022be78c-a41b-49b0-9d40-fae1ebb4a698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:47.714497 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1301/2617] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Oct 24 13:45:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:47.726715 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c4e3e-cde9-4bc6-9062-4847b1774b87 service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:47.736980 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c4e3e-cde9-4bc6-9062-4847b1774b87 service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:47.742540 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ab202-1840-427e-b836-55219918de2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:47.743112 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab202-1840-427e-b836-55219918de2c service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:47.743404 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab202-1840-427e-b836-55219918de2c service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:47.743672 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab202-1840-427e-b836-55219918de2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:47.751343 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ab202-1840-427e-b836-55219918de2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:47.752932 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ab202-1840-427e-b836-55219918de2c service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:47.760753 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1317/2618] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:45:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:45:48.279836 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed9d71-a0b6-4c22-8ba0-d37347e4efe3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:48.280327 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed9d71-a0b6-4c22-8ba0-d37347e4efe3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:48.280429 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed9d71-a0b6-4c22-8ba0-d37347e4efe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:48.299694 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dbed9d71-a0b6-4c22-8ba0-d37347e4efe3 None None] MFA Rules not processed for user `038f2ef649844de297237a0c1ec652aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:48.300890 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dbed9d71-a0b6-4c22-8ba0-d37347e4efe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:48.307888 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1302/2619] 10.222.0.48 () {58 vars in 967 bytes} [Mon Oct 24 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:48.320407 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ab202-1840-427e-b836-55219918de2c service swift] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:48.330904 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ab202-1840-427e-b836-55219918de2c service swift] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:48.336405 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e6550-c47e-41fa-abe2-111de4d25b2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '035a080de9d54a56af29cc56b33a1ddc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:48.336981 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e6550-c47e-41fa-abe2-111de4d25b2d service swift] REQUEST_METHOD: `GET` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:48.337255 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e6550-c47e-41fa-abe2-111de4d25b2d service swift] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:48.337522 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e6550-c47e-41fa-abe2-111de4d25b2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:48.345294 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e6550-c47e-41fa-abe2-111de4d25b2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:48.346845 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e6550-c47e-41fa-abe2-111de4d25b2d service swift] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:48.355089 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1318/2620] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Oct 24 13:45:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:45:50.963889 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed9d71-a0b6-4c22-8ba0-d37347e4efe3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:51.024260 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed9d71-a0b6-4c22-8ba0-d37347e4efe3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:51.028929 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:51.029585 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:51.029863 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:51.030133 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] PATH_INFO: `/v3/users/a0b551495dca4982bef194a97c707fb5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:51.031276 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b551495dca4982bef194a97c707fb5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:51.033248 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:51.249468 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b551495dca4982bef194a97c707fb5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.256659 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b551495dca4982bef194a97c707fb5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.268733 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b551495dca4982bef194a97c707fb5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.272318 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:51.356173 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b551495dca4982bef194a97c707fb5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.356814 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b551495dca4982bef194a97c707fb5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.363560 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1303/2621] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Oct 24 13:45:50 2022] DELETE /identity/v3/users/a0b551495dca4982bef194a97c707fb5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:45:51.368509 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708faa-665e-4c67-b10e-8694a64a3b36 None None] REQUEST_METHOD: `POST` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:51.368993 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708faa-665e-4c67-b10e-8694a64a3b36 None None] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:51.369111 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708faa-665e-4c67-b10e-8694a64a3b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:51.387731 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.auth.core [None req-d7708faa-665e-4c67-b10e-8694a64a3b36 None None] MFA Rules not processed for user `742bb1241cf54598b570a2fd87179834`. Rule list: `[]` (Enabled: `True`). {{(pid=81603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:45:51.389017 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d7708faa-665e-4c67-b10e-8694a64a3b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:51.447493 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1319/2622] 10.222.0.48 () {58 vars in 958 bytes} [Mon Oct 24 13:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:45:51.457086 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:51.458503 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:51.515025 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6432a98-b409-4dfb-8353-98dfa731be8a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:51.518448 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d4ca1062fe496dbe16d6cd1bcac1a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:51.519033 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:51.519305 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:51.519571 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:51.528996 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:51.530386 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:51.540244 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1304/2623] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:45:51.763007 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7708faa-665e-4c67-b10e-8694a64a3b36 None None] Authenticating user token {{(pid=81603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:45:51.764455 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.fernet_utils [None req-d7708faa-665e-4c67-b10e-8694a64a3b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:45:51.777537 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7708faa-665e-4c67-b10e-8694a64a3b36 None None] Validating token access rules against request {{(pid=81603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:45:51.782608 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742bb1241cf54598b570a2fd87179834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5f7bf50f5a94c7694720e6be18c9de2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:45:51.783324 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:45:51.784195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:45:51.784195 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] PATH_INFO: `/v3/projects/0cf2f5ececed49adb39332338164ec3f` {{(pid=81603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:45:51.785288 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf2f5ececed49adb39332338164ec3f)` {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:45:51.787234 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] RBAC: Authorization granted {{(pid=81603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:45:51.877691 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:45:51.886983 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf2f5ececed49adb39332338164ec3f', 'request_id': 'req-181f001d-e176-4dd1-bd26-e46352e36bd8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.966820 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf2f5ececed49adb39332338164ec3f', 'request_id': 'req-181f001d-e176-4dd1-bd26-e46352e36bd8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.973557 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf2f5ececed49adb39332338164ec3f', 'request_id': 'req-181f001d-e176-4dd1-bd26-e46352e36bd8'} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.984175 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] The token cache is being invalidate because project 0cf2f5ececed49adb39332338164ec3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:45:51.984492 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.985101 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG keystone.notifications [None req-181f001d-e176-4dd1-bd26-e46352e36bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:45:51.986553 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: [pid: 81603|app: 0|req: 1320/2624] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Oct 24 13:45:51 2022] DELETE /identity/v3/projects/0cf2f5ececed49adb39332338164ec3f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:46:01.535479 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=28 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:46:01.981185 dv-cinder-862491-1-iscsi devstack@keystone.service[81603]: DEBUG dbcounter [-] [81603] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:48:40.906262 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:48:40.907675 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:40.976726 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6f76c-58fd-4039-ad37-380978d533e8 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:48:41.035167 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c270f8-4e92-4ac0-9a21-d9767339ce3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dd54892919e41f9b1f2ad734dc554f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7508e18ad6cf4b14afba4f86574a19e0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:48:41.035768 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c270f8-4e92-4ac0-9a21-d9767339ce3b service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:48:41.036047 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c270f8-4e92-4ac0-9a21-d9767339ce3b service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:48:41.036347 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c270f8-4e92-4ac0-9a21-d9767339ce3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:48:41.038057 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-57c270f8-4e92-4ac0-9a21-d9767339ce3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:48:41.049622 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c270f8-4e92-4ac0-9a21-d9767339ce3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:48:41.051176 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c270f8-4e92-4ac0-9a21-d9767339ce3b service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:48:41.060866 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1305/2625] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Oct 24 13:48:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:48:51.056654 dv-cinder-862491-1-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=58 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}